Merge lp:~jameinel/bzr-fastimport/mooloolaba into lp:~bzr/bzr-fastimport/fastimport.dev
Proposed by
John A Meinel
Status: | Superseded |
---|---|
Proposed branch: | lp:~jameinel/bzr-fastimport/mooloolaba |
Merge into: | lp:~bzr/bzr-fastimport/fastimport.dev |
To merge this branch: | bzr merge lp:~jameinel/bzr-fastimport/mooloolaba |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Bazaar Developers | Pending | ||
Review via email: mp+14894@code.launchpad.net |
This proposal has been superseded by a proposal from 2009-11-29.
To post a comment you must log in.
This branch is in --2a format, but trunk isn't, so this request may fail.
Anyway, this is just a rollup of the stuff I worked on in Mooloolaba. Stuff like:
1) Memory reduction by referencing texts as strings rather than lines. This alone caused my peak qt import memory to go to 1.5GB from >2GB (I don't know how big, because I go OOM @2GB on my 32-bit machine).
2) Memory reduction by not caching all texts in memory. Instead, dump them into temp files. 'small' blob files are all collected into a single temp file, everything else gets a unique file that will be deleted when the last reference goes away. This got me down to between 500MB => 900MB during the import.
3) Switch the code that adds things to the repo to use CommitBuilder. It cheats a little bit here and there to get around deficiencies in the api. Namely, CommitBuilder wants to start_write_group and commit_write_group for every build, and we wanted to work around that.
I'm not sure what else is included, but that is the basics.