jeudi 30 juin 2016

Large SQL query eats memory simply by retrieving data

I have a PHP script which calls a "large" SQL query - "large" in term of retrieved records (close to 1 million). At first it worked fine and now after the database reached this size I've been getting a "Killed." message after a while. A quick check showed that the PHP script was simply out of memory.

Originally I thought this was an issue with the code, that I somehow created an array that grows as more data is retrieved from the query, but that is not the case. After commenting all the code except

for ($i = 0; $i < $result->num_rows; $i++) {
            $data = $result->fetch_assoc();
            echo "Using ", memory_get_peak_usage(1), " bytes of ram.n";
}

it still shows the memory increasing over time until - again, the script is "Killed.". Any idea how to retrieve records from the database in a way that it will not constantly increase the memory footprint? or is there no way around it and I just need to increase the RAM available for the script?

Thanks!

Aucun commentaire:

Enregistrer un commentaire