[wp-trac] Re: [WordPress Trac] #8999: Completely New LiveJournal
Importer
WordPress Trac
wp-trac at lists.automattic.com
Mon Feb 9 08:16:54 GMT 2009
#8999: Completely New LiveJournal Importer
-------------------------------------+--------------------------------------
Reporter: beaulebens | Owner:
Type: task (blessed) | Status: reopened
Priority: normal | Milestone: 2.8
Component: Import | Version: 2.7
Severity: normal | Resolution:
Keywords: needs-testing has-patch |
-------------------------------------+--------------------------------------
Comment (by westi):
Replying to [comment:38 beaulebens]:
> OK - I've just run a full import (3,750 posts, 200,000 comments) and it
took just over 6 HOURS to import (previous, file-based version too just
under 3 hours for the same journal). The comments really kill this process
-- re-threading takes 3 hours because querying the comments is super slow
since I'm using "spare" fields that aren't indexed.
>
Cool. Although 6 hours is a little long ;-)
> Importing a smaller journal goes quickly and without problem.
>
> 2 options here to speed it up for big journals:
>
> 1. Before re-threading, ALTER TABLE ADD INDEX on the 3 fields that are
used. That seems to reduce things down to literally a couple of minutes,
then I can DROP INDEX when I'm done, or
> 2. Use a temporary table in MySQL that's optimized for what I'm trying
to do (but then I'd have all sorts of custom code that operated outside
the normal comments API).
>
> I'd lean towards the first option, but I don't know how that jives with
the general approach of core code?
>
What are the extra fields you are using.
Is there any reason why they couldn't just be indexed anyway?
I think we should go with the speediest solution - people won't want to
wait 6 hours for there import to complete!
--
Ticket URL: <http://trac.wordpress.org/ticket/8999#comment:39>
WordPress Trac <http://trac.wordpress.org/>
WordPress blogging software
More information about the wp-trac
mailing list