In a few days, there was a Symfony command that iterates table rows making some logic to delete it in the final step. Except that the script stops causing an exhausted memory size.
[2020-02-20 10:36:39] php.CRITICAL: Fatal Error: Allowed memory size of 734003200 bytes exhausted (tried to allocate 135168 bytes) {"exception":"[object] (Symfony\\Component\\Debug\\Exception\\OutOfMemoryException(code: 0): Error: Allowed memory size of 734003200 bytes exhausted (tried to allocate 135168 bytes) at /var/www/myfm/vendor/doctrine/dbal/lib/Doctrine/DBAL/SQLParserUtils. php:136)"} []
Context
loop for each row, we try to remove (using doctrine & monolog to trace every transaction) it and all the other rows in association from other tables (to the extent of 20 tables).
Existing prod configuration
monolog:
handlers:
main:
type: fingers_crossed
action_level: error
handler: nested
nested:
type: stream
path: "php://stderr"
level: error
notice:
type: stream
path: "php://stdout"
level: notice
First, let’s explain what refers to the above configuration for more understanding.
The Fingers_crossed monolog handler works as logs stake, until the min_level, is reached to give hand to the nested handler to take the responsibility of logging and then passing the record to the next handler till the end.
Overall, it’s clearly evident that if the record level never reaches the ERROR level, the stack will register a lot of records which will fill the memory.
Doctrine profiling and logging are factors that go into the equation and could alter the expected result. By default, the doctrine profiling and logging take the value of “kernel.debug” which depends on the environment.
Always think to clear cache whenever you change the configuration or command options .
In case you want to disable the doctrine profiling and logging, try the snippet below :
doctrine: dbal: default_connection: default connections: default: dbname: inferom user: inferom password: inferom host: inferom profiling: false logging: false
But this will not resolve the memory leak problem because the headache is hidden behind the monolog.
Use case
commenting all lines contains logging reduce memory usage by 90% compared to before which means that something is going wrong.
Solution
The problem was living in the monolog which could be resolved by changing some configurations.
the buffer_size monolog configuration limits the stake size. how could resolve the issue?
In fact, I mentioned above, the Fingers_cross handler, store the records(logs) into a stack(array). Pushing the buffer_size will automatically release the stack when the buffer reaches the buffer_size value and so on.
monolog: handlers: main: type: fingers_crossed action_level: error handler: nested buffer_size: 100 nested: type: stream path: "php://stderr" level: error notice: type: stream path: "php://stdout" level: notice
reassuring of a no overload in the stack the causes the leaked memory.
I hope this feedback was helpful and relevant.