[wp-trac] Re: [WordPress Trac] #8999: Completely New LiveJournal
Importer
WordPress Trac
wp-trac at lists.automattic.com
Sun Feb 8 20:14:06 GMT 2009
#8999: Completely New LiveJournal Importer
-------------------------------------+--------------------------------------
Reporter: beaulebens | Owner:
Type: task (blessed) | Status: reopened
Priority: normal | Milestone: 2.8
Component: Import | Version: 2.7
Severity: normal | Resolution:
Keywords: needs-testing has-patch |
-------------------------------------+--------------------------------------
Comment (by beaulebens):
OK - I've just run a full import (3,750 posts, 200,000 comments) and it
took just over 6 HOURS to import (previous, file-based version too just
under 3 hours for the same journal). The comments really kill this process
-- re-threading takes 3 hours because querying the comments is super slow
since I'm using "spare" fields that aren't indexed.
Importing a smaller journal goes quickly and without problem.
2 options here to speed it up for big journals:
1. Before re-threading, ALTER TABLE ADD INDEX on the 3 fields that are
used. That seems to reduce things down to literally a couple of minutes,
then I can DROP INDEX when I'm done, or
2. Use a temporary table in MySQL that's optimized for what I'm trying to
do (but then I'd have all sorts of custom code that operated outside the
normal comments API).
I'd lean towards the first option, but I don't know how that jives with
the general approach of core code?
Comments? Ryan?
--
Ticket URL: <http://trac.wordpress.org/ticket/8999#comment:38>
WordPress Trac <http://trac.wordpress.org/>
WordPress blogging software
More information about the wp-trac
mailing list