[wp-hackers] Incremental rows retrieval

[Justas Butkus]

Hi all.

Is there a chance, without bypassing `wpdb`, to get resulting records 
from `SELECT` query in an incremental manner, instead of current 
behaviour when `wpdb` pre-fetches them all?

Let's assume I have a query which might result in tens of thousands of rows.
Holding it all in memory (like wpdb currently does when I execute 
`SELECT`) would require a lot of memory.
If I were able to process them incrementally memory consumption may be 
low, as they are fed to external resource and then discarded.

I have considered following alternative: using multiple queries and 
adding `WHERE primary_key > $last_value ORDER BY primary_key ASC` to 
subsequent ones.
It seems to be the best solution given MySQL index optimisation.
The problem here is consistency - if rows has cross dependency there is 
no guarantee that all relates rows will fall into a single interval.

Though - is there a chance to get records in an incremental manner 
without breaking wpdb interfaces?


-- 
Regards,
Justas Butkus