Merge lp:~jelmer/launchpad/sync-tbz2 into lp:launchpad/db-devel
- sync-tbz2
- Merge into db-devel
Proposed by
Jelmer Vernooij
| Status: | Merged |
|---|---|
| Approved by: | Jelmer Vernooij |
| Approved revision: | not available |
| Merged at revision: | not available |
| Proposed branch: | lp:~jelmer/launchpad/sync-tbz2 |
| Merge into: | lp:launchpad/db-devel |
| Diff against target: |
916 lines (+303/-297) 7 files modified
lib/canonical/launchpad/scripts/logger.py (+1/-1) lib/lp/soyuz/scripts/ftpmaster.py (+27/-27) lib/lp/soyuz/scripts/tests/sync_source_home/Debian_incoming_main_Sources (+28/-0) lib/lp/soyuz/scripts/tests/sync_source_home/sample1_1.0-1.dsc (+37/-0) lib/lp/soyuz/scripts/tests/test_sync_source.py (+105/-28) scripts/ftpmaster-tools/dak_utils.py (+3/-153) scripts/ftpmaster-tools/sync-source.py (+102/-88) |
| To merge this branch: | bzr merge lp:~jelmer/launchpad/sync-tbz2 |
| Related bugs: |
| Reviewer | Review Type | Date Requested | Status |
|---|---|---|---|
| Guilherme Salgado (community) | code | Approve | |
|
Review via email:
|
|||
Commit message
Fixes support for syncing packages in the the version 3 source format.
Description of the change
To post a comment you must log in.
Revision history for this message
| Jelmer Vernooij (jelmer) wrote : | # |
Revision history for this message
| Guilherme Salgado (salgado) wrote : | # |
Download full text (10.4 KiB)
Hi Jelmer,
I have just a few comments/questions below, but this looks good. And
thanks a lot for getting rid of the untested/duplicated code.
review approve code
On Thu, 2010-02-04 at 16:24 +0000, Jelmer Vernooij wrote:
> === modified file 'lib/canonical/
> --- lib/canonical/
> +++ lib/canonical/
[...]
> @@ -847,19 +854,14 @@
> Return the fetched filename if it was present in Librarian or None
> if it wasn't.
> """
> - # XXX cprov 2007-01-10 bug=78683: Looking for files within ubuntu
> - # only. It doesn't affect the usual sync-source procedure. However
> - # it needs to be revisited for derivation, we probably need
> - # to pass the target distribution in order to make proper lookups.
> - ubuntu = getUtility(
Does this mean the bug above will be fixed too?
> try:
> - libraryfilealias = ubuntu.
> + libraryfilealias = self.todistro.
> filename, source=True, binary=False)
> except NotFoundError:
> return None
>
> - self.debug(
> - "\t%s: already in distro - downloading from librarian" %
> + self.logger.info(
> + "%s: already in distro - downloading from librarian" %
> filename)
>
> output_file = open(filename, 'w')
> @@ -871,23 +873,23 @@
> """Try to fetch files from Librarian.
>
> It raises SyncSourceError if anything else then an
> - 'orig.tar.gz' was found in Librarian.
> - Return a boolean indicating whether or not the 'orig.tar.gz' is
> - required in the upload.
> + orig tarball was found in Librarian.
> + Return the names of the files retrieved from the librarian.
> """
> - orig_filename = None
> + retrieved = []
> for filename in self.files.keys():
> if not self.fetchFileF
> continue
> + file_type = determine_
> # set the return code if an orig was, in fact,
> # fetched from Librarian
> - if filename.
> - orig_filename = filename
> - else:
> + if not file_type in (SourcePackageF
> + SourcePackageFi
The indentation above is a bit odd; the two elements of the tuple should
be lined up.
> raise SyncSourceError(
> - 'Oops, only orig.tar.gz can be retrieved from librarian')
> + 'Oops, only orig tarball can be retrieved from librarian.')
> + retrieved.
>
> - return orig_filename
> + return retrieved
>
> def fetchSyncFiles(
> """Fetch files from the original sync source.
>
> === modified file 'scripts/
> --- scripts/
review:
Approve
(code)
Revision history for this message
| Jelmer Vernooij (jelmer) wrote : | # |
Hi Guilherme,
Thanks for the review.
On Fri, 2010-02-05 at 17:48 +0000, Guilherme Salgado wrote:
> On Thu, 2010-02-04 at 16:24 +0000, Jelmer Vernooij wrote:
> > === modified file 'lib/canonical/
> > --- lib/canonical/
> > +++ lib/canonical/
> [...]
> > @@ -847,19 +854,14 @@
> > Return the fetched filename if it was present in Librarian or None
> > if it wasn't.
> > """
> > - # XXX cprov 2007-01-10 bug=78683: Looking for files within ubuntu
> > - # only. It doesn't affect the usual sync-source procedure. However
> > - # it needs to be revisited for derivation, we probably need
> > - # to pass the target distribution in order to make proper lookups.
> > - ubuntu = getUtility(
> Does this mean the bug above will be fixed too?
Partially - bug 78683 is about sync-source only working with Ubuntu, and
this change gets us further in that direction but there are still some
places where we hardcode 'ubuntu' in scripts/
I've addressed the style issue you've raised and I'll ec2 land after I
finish testing on dogfood.
Cheers,
Jelmer
Preview Diff
[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
| 1 | === modified file 'lib/canonical/launchpad/scripts/logger.py' | |||
| 2 | --- lib/canonical/launchpad/scripts/logger.py 2009-10-16 18:07:01 +0000 | |||
| 3 | +++ lib/canonical/launchpad/scripts/logger.py 2010-02-11 16:55:27 +0000 | |||
| 4 | @@ -5,7 +5,7 @@ | |||
| 5 | 5 | 5 | ||
| 6 | 6 | """Logging setup for scripts. | 6 | """Logging setup for scripts. |
| 7 | 7 | 7 | ||
| 9 | 8 | Don't import from this module. Import it from canonical.scripts. | 8 | Don't import from this module. Import it from canonical.launchpad.scripts. |
| 10 | 9 | 9 | ||
| 11 | 10 | Parts of this may be moved into canonical.launchpad somewhere if it is | 10 | Parts of this may be moved into canonical.launchpad somewhere if it is |
| 12 | 11 | to be used for non-script stuff. | 11 | to be used for non-script stuff. |
| 13 | 12 | 12 | ||
| 14 | === modified file 'lib/lp/soyuz/scripts/ftpmaster.py' | |||
| 15 | --- lib/lp/soyuz/scripts/ftpmaster.py 2009-12-13 11:55:40 +0000 | |||
| 16 | +++ lib/lp/soyuz/scripts/ftpmaster.py 2010-02-11 16:55:27 +0000 | |||
| 17 | @@ -36,11 +36,14 @@ | |||
| 18 | 36 | from canonical.launchpad.interfaces.launchpad import ILaunchpadCelebrities | 36 | from canonical.launchpad.interfaces.launchpad import ILaunchpadCelebrities |
| 19 | 37 | from canonical.launchpad.interfaces.librarian import ILibraryFileAliasSet | 37 | from canonical.launchpad.interfaces.librarian import ILibraryFileAliasSet |
| 20 | 38 | from canonical.launchpad.webapp.interfaces import NotFoundError | 38 | from canonical.launchpad.webapp.interfaces import NotFoundError |
| 21 | 39 | from lp.archiveuploader.utils import ( | ||
| 22 | 40 | determine_source_file_type) | ||
| 23 | 39 | from lp.registry.interfaces.distribution import IDistributionSet | 41 | from lp.registry.interfaces.distribution import IDistributionSet |
| 24 | 40 | from lp.registry.interfaces.series import SeriesStatus | 42 | from lp.registry.interfaces.series import SeriesStatus |
| 25 | 41 | from lp.registry.interfaces.person import IPersonSet | 43 | from lp.registry.interfaces.person import IPersonSet |
| 26 | 42 | from lp.registry.interfaces.pocket import ( | 44 | from lp.registry.interfaces.pocket import ( |
| 27 | 43 | PackagePublishingPocket, pocketsuffix) | 45 | PackagePublishingPocket, pocketsuffix) |
| 28 | 46 | from lp.registry.interfaces.sourcepackage import SourcePackageFileType | ||
| 29 | 44 | from lp.soyuz.interfaces.binarypackagename import IBinaryPackageNameSet | 47 | from lp.soyuz.interfaces.binarypackagename import IBinaryPackageNameSet |
| 30 | 45 | from lp.soyuz.interfaces.binarypackagerelease import IBinaryPackageReleaseSet | 48 | from lp.soyuz.interfaces.binarypackagerelease import IBinaryPackageReleaseSet |
| 31 | 46 | from lp.soyuz.interfaces.publishing import PackagePublishingStatus | 49 | from lp.soyuz.interfaces.publishing import PackagePublishingStatus |
| 32 | @@ -803,12 +806,14 @@ | |||
| 33 | 803 | pocket_chroot.chroot.open() | 806 | pocket_chroot.chroot.open() |
| 34 | 804 | copy_and_close(pocket_chroot.chroot, local_file) | 807 | copy_and_close(pocket_chroot.chroot, local_file) |
| 35 | 805 | 808 | ||
| 36 | 809 | |||
| 37 | 806 | class SyncSourceError(Exception): | 810 | class SyncSourceError(Exception): |
| 38 | 807 | """Raised when an critical error occurs inside SyncSource. | 811 | """Raised when an critical error occurs inside SyncSource. |
| 39 | 808 | 812 | ||
| 40 | 809 | The entire procedure should be aborted in order to avoid unknown problems. | 813 | The entire procedure should be aborted in order to avoid unknown problems. |
| 41 | 810 | """ | 814 | """ |
| 42 | 811 | 815 | ||
| 43 | 816 | |||
| 44 | 812 | class SyncSource: | 817 | class SyncSource: |
| 45 | 813 | """Sync Source procedure helper class. | 818 | """Sync Source procedure helper class. |
| 46 | 814 | 819 | ||
| 47 | @@ -818,20 +823,22 @@ | |||
| 48 | 818 | 'aptMD5Sum' is provided as a classmethod during the integration time. | 823 | 'aptMD5Sum' is provided as a classmethod during the integration time. |
| 49 | 819 | """ | 824 | """ |
| 50 | 820 | 825 | ||
| 52 | 821 | def __init__(self, files, origin, debug, downloader): | 826 | def __init__(self, files, origin, logger, downloader, todistro): |
| 53 | 822 | """Store local context. | 827 | """Store local context. |
| 54 | 823 | 828 | ||
| 55 | 824 | files: a dictionary where the keys are the filename and the | 829 | files: a dictionary where the keys are the filename and the |
| 56 | 825 | value another dictionary with the file informations. | 830 | value another dictionary with the file informations. |
| 57 | 826 | origin: a dictionary similar to 'files' but where the values | 831 | origin: a dictionary similar to 'files' but where the values |
| 58 | 827 | contain information for download files to be synchronized | 832 | contain information for download files to be synchronized |
| 60 | 828 | debug: a debug function, 'debug(message)' | 833 | logger: a logger |
| 61 | 829 | downloader: a callable that fetchs URLs, 'downloader(url, destination)' | 834 | downloader: a callable that fetchs URLs, 'downloader(url, destination)' |
| 62 | 835 | todistro: target distribution object | ||
| 63 | 830 | """ | 836 | """ |
| 64 | 831 | self.files = files | 837 | self.files = files |
| 65 | 832 | self.origin = origin | 838 | self.origin = origin |
| 67 | 833 | self.debug = debug | 839 | self.logger = logger |
| 68 | 834 | self.downloader = downloader | 840 | self.downloader = downloader |
| 69 | 841 | self.todistro = todistro | ||
| 70 | 835 | 842 | ||
| 71 | 836 | @classmethod | 843 | @classmethod |
| 72 | 837 | def generateMD5Sum(self, filename): | 844 | def generateMD5Sum(self, filename): |
| 73 | @@ -847,19 +854,14 @@ | |||
| 74 | 847 | Return the fetched filename if it was present in Librarian or None | 854 | Return the fetched filename if it was present in Librarian or None |
| 75 | 848 | if it wasn't. | 855 | if it wasn't. |
| 76 | 849 | """ | 856 | """ |
| 77 | 850 | # XXX cprov 2007-01-10 bug=78683: Looking for files within ubuntu | ||
| 78 | 851 | # only. It doesn't affect the usual sync-source procedure. However | ||
| 79 | 852 | # it needs to be revisited for derivation, we probably need | ||
| 80 | 853 | # to pass the target distribution in order to make proper lookups. | ||
| 81 | 854 | ubuntu = getUtility(IDistributionSet)['ubuntu'] | ||
| 82 | 855 | try: | 857 | try: |
| 84 | 856 | libraryfilealias = ubuntu.getFileByName( | 858 | libraryfilealias = self.todistro.getFileByName( |
| 85 | 857 | filename, source=True, binary=False) | 859 | filename, source=True, binary=False) |
| 86 | 858 | except NotFoundError: | 860 | except NotFoundError: |
| 87 | 859 | return None | 861 | return None |
| 88 | 860 | 862 | ||
| 91 | 861 | self.debug( | 863 | self.logger.info( |
| 92 | 862 | "\t%s: already in distro - downloading from librarian" % | 864 | "%s: already in distro - downloading from librarian" % |
| 93 | 863 | filename) | 865 | filename) |
| 94 | 864 | 866 | ||
| 95 | 865 | output_file = open(filename, 'w') | 867 | output_file = open(filename, 'w') |
| 96 | @@ -871,23 +873,23 @@ | |||
| 97 | 871 | """Try to fetch files from Librarian. | 873 | """Try to fetch files from Librarian. |
| 98 | 872 | 874 | ||
| 99 | 873 | It raises SyncSourceError if anything else then an | 875 | It raises SyncSourceError if anything else then an |
| 103 | 874 | 'orig.tar.gz' was found in Librarian. | 876 | orig tarball was found in Librarian. |
| 104 | 875 | Return a boolean indicating whether or not the 'orig.tar.gz' is | 877 | Return the names of the files retrieved from the librarian. |
| 102 | 876 | required in the upload. | ||
| 105 | 877 | """ | 878 | """ |
| 107 | 878 | orig_filename = None | 879 | retrieved = [] |
| 108 | 879 | for filename in self.files.keys(): | 880 | for filename in self.files.keys(): |
| 109 | 880 | if not self.fetchFileFromLibrarian(filename): | 881 | if not self.fetchFileFromLibrarian(filename): |
| 110 | 881 | continue | 882 | continue |
| 111 | 883 | file_type = determine_source_file_type(filename) | ||
| 112 | 882 | # set the return code if an orig was, in fact, | 884 | # set the return code if an orig was, in fact, |
| 113 | 883 | # fetched from Librarian | 885 | # fetched from Librarian |
| 117 | 884 | if filename.endswith("orig.tar.gz"): | 886 | if not file_type in (SourcePackageFileType.ORIG_TARBALL, |
| 118 | 885 | orig_filename = filename | 887 | SourcePackageFileType.COMPONENT_ORIG_TARBALL): |
| 116 | 886 | else: | ||
| 119 | 887 | raise SyncSourceError( | 888 | raise SyncSourceError( |
| 121 | 888 | 'Oops, only orig.tar.gz can be retrieved from librarian') | 889 | 'Oops, only orig tarball can be retrieved from librarian.') |
| 122 | 890 | retrieved.append(filename) | ||
| 123 | 889 | 891 | ||
| 125 | 890 | return orig_filename | 892 | return retrieved |
| 126 | 891 | 893 | ||
| 127 | 892 | def fetchSyncFiles(self): | 894 | def fetchSyncFiles(self): |
| 128 | 893 | """Fetch files from the original sync source. | 895 | """Fetch files from the original sync source. |
| 129 | @@ -896,21 +898,19 @@ | |||
| 130 | 896 | """ | 898 | """ |
| 131 | 897 | dsc_filename = None | 899 | dsc_filename = None |
| 132 | 898 | for filename in self.files.keys(): | 900 | for filename in self.files.keys(): |
| 133 | 901 | file_type = determine_source_file_type(filename) | ||
| 134 | 902 | if file_type == SourcePackageFileType.DSC: | ||
| 135 | 903 | dsc_filename = filename | ||
| 136 | 899 | if os.path.exists(filename): | 904 | if os.path.exists(filename): |
| 137 | 905 | self.logger.info(" - <%s: cached>" % (filename)) | ||
| 138 | 900 | continue | 906 | continue |
| 140 | 901 | self.debug( | 907 | self.logger.info( |
| 141 | 902 | " - <%s: downloading from %s>" % | 908 | " - <%s: downloading from %s>" % |
| 142 | 903 | (filename, self.origin["url"])) | 909 | (filename, self.origin["url"])) |
| 143 | 904 | download_f = ("%s%s" % (self.origin["url"], | 910 | download_f = ("%s%s" % (self.origin["url"], |
| 144 | 905 | self.files[filename]["remote filename"])) | 911 | self.files[filename]["remote filename"])) |
| 145 | 906 | sys.stdout.flush() | 912 | sys.stdout.flush() |
| 146 | 907 | self.downloader(download_f, filename) | 913 | self.downloader(download_f, filename) |
| 147 | 908 | # only set the dsc_filename if the DSC was really downloaded. | ||
| 148 | 909 | # this loop usually includes the other files for the upload, | ||
| 149 | 910 | # DIFF and ORIG. | ||
| 150 | 911 | if filename.endswith(".dsc"): | ||
| 151 | 912 | dsc_filename = filename | ||
| 152 | 913 | |||
| 153 | 914 | return dsc_filename | 914 | return dsc_filename |
| 154 | 915 | 915 | ||
| 155 | 916 | def checkDownloadedFiles(self): | 916 | def checkDownloadedFiles(self): |
| 156 | 917 | 917 | ||
| 157 | === modified file 'lib/lp/soyuz/scripts/tests/sync_source_home/Debian_incoming_main_Sources' | |||
| 158 | --- lib/lp/soyuz/scripts/tests/sync_source_home/Debian_incoming_main_Sources 2008-02-14 12:38:50 +0000 | |||
| 159 | +++ lib/lp/soyuz/scripts/tests/sync_source_home/Debian_incoming_main_Sources 2010-02-11 16:55:27 +0000 | |||
| 160 | @@ -10,3 +10,31 @@ | |||
| 161 | 10 | fc1464e5985b962a042d5354452f361d 164 bar_1.0.orig.tar.gz | 10 | fc1464e5985b962a042d5354452f361d 164 bar_1.0.orig.tar.gz |
| 162 | 11 | a259bf88aedcdeded4e5d94ad4af3b4a 610 bar_1.0-1.diff.gz | 11 | a259bf88aedcdeded4e5d94ad4af3b4a 610 bar_1.0-1.diff.gz |
| 163 | 12 | 12 | ||
| 164 | 13 | Format: 3.0 (quilt) | ||
| 165 | 14 | Package: sample1 | ||
| 166 | 15 | Binary: sample1 | ||
| 167 | 16 | Architecture: all | ||
| 168 | 17 | Version: 1.0-1 | ||
| 169 | 18 | Maintainer: Raphael Hertzog <hertzog@debian.org> | ||
| 170 | 19 | Standards-Version: 3.8.1 | ||
| 171 | 20 | Checksums-Sha1: | ||
| 172 | 21 | 724e3cdad5f4af13383e5d71f525465add1ce897 1741 sample1_1.0-1.dsc | ||
| 173 | 22 | 296cb89762668772c3ae4799bd9ce7973f4154b2 171 sample1_1.0.orig-component1.tar.bz2 | ||
| 174 | 23 | ccb198be27c065880696cbeb6fb5aebf190c8544 157 sample1_1.0.orig-component2.tar.lzma | ||
| 175 | 24 | ea3c0c6ee23410a9c454d9b35681a80460343456 201 sample1_1.0.orig-component3.tar.gz | ||
| 176 | 25 | c62b2496d3ff0c64dcf6488c95e1d5cbc9f5f30d 201 sample1_1.0.orig.tar.gz | ||
| 177 | 26 | b3c92ffaabc2e3e95e5f74e1de94351fddcb065c 1327 sample1_1.0-1.debian.tar.gz | ||
| 178 | 27 | Checksums-Sha256: | ||
| 179 | 28 | d5390b56d3d5ea301bb565487eded8a6739d29ea5d680389f35b87b30f846412 1741 sample1_1.0-1.dsc | ||
| 180 | 29 | a0871b73922a228ef8c1f4c1a7e1aef94d8a59d919218242655b7faae352d5c9 171 sample1_1.0.orig-component1.tar.bz2 | ||
| 181 | 30 | 39963ed90d70e918a6cae8d73c81f42dcb7f0dc283a4715f032a1e28b98d1a25 157 sample1_1.0.orig-component2.tar.lzma | ||
| 182 | 31 | dc174acae810faf5f9a679e042f480905622fe4a854510dad6e200dbe6381c8d 201 sample1_1.0.orig-component3.tar.gz | ||
| 183 | 32 | 9c0f0b42f6283614b204a47600cb9b0d8b19dfa4cb43c69e6a0a40f3d1064e0e 201 sample1_1.0.orig.tar.gz | ||
| 184 | 33 | 6864ceeb315ad7b6a59636a0251525cbd8e9b76c6e934f3746271d4ebce5da3c 1327 sample1_1.0-1.debian.tar.gz | ||
| 185 | 34 | Files: | ||
| 186 | 35 | aea108574786aa670dc5ef26dc225be2 1741 sample1_1.0-1.dsc | ||
| 187 | 36 | 8a90ee77cf3c68c32bdb5c55d40d06f4 171 sample1_1.0.orig-component1.tar.bz2 | ||
| 188 | 37 | 3535d4766df47e531372802a0c2ddb82 157 sample1_1.0.orig-component2.tar.lzma | ||
| 189 | 38 | 673b42066e8e858208b5d0c7a006c5fb 201 sample1_1.0.orig-component3.tar.gz | ||
| 190 | 39 | 9047a93bf12a7c749045e9bc9ff9ee99 201 sample1_1.0.orig.tar.gz | ||
| 191 | 40 | 9404bc99cdb06876a988d4c2262c073e 1327 sample1_1.0-1.debian.tar.gz | ||
| 192 | 13 | 41 | ||
| 193 | === added file 'lib/lp/soyuz/scripts/tests/sync_source_home/sample1_1.0-1.debian.tar.gz' | |||
| 194 | 14 | Binary files lib/lp/soyuz/scripts/tests/sync_source_home/sample1_1.0-1.debian.tar.gz 1970-01-01 00:00:00 +0000 and lib/lp/soyuz/scripts/tests/sync_source_home/sample1_1.0-1.debian.tar.gz 2010-02-11 16:55:27 +0000 differ | 42 | Binary files lib/lp/soyuz/scripts/tests/sync_source_home/sample1_1.0-1.debian.tar.gz 1970-01-01 00:00:00 +0000 and lib/lp/soyuz/scripts/tests/sync_source_home/sample1_1.0-1.debian.tar.gz 2010-02-11 16:55:27 +0000 differ |
| 195 | === added file 'lib/lp/soyuz/scripts/tests/sync_source_home/sample1_1.0-1.dsc' | |||
| 196 | --- lib/lp/soyuz/scripts/tests/sync_source_home/sample1_1.0-1.dsc 1970-01-01 00:00:00 +0000 | |||
| 197 | +++ lib/lp/soyuz/scripts/tests/sync_source_home/sample1_1.0-1.dsc 2010-02-11 16:55:27 +0000 | |||
| 198 | @@ -0,0 +1,37 @@ | |||
| 199 | 1 | -----BEGIN PGP SIGNED MESSAGE----- | ||
| 200 | 2 | Hash: SHA1 | ||
| 201 | 3 | |||
| 202 | 4 | Format: 3.0 (quilt) | ||
| 203 | 5 | Source: sample1 | ||
| 204 | 6 | Binary: sample1 | ||
| 205 | 7 | Architecture: all | ||
| 206 | 8 | Version: 1.0-1 | ||
| 207 | 9 | Maintainer: Raphael Hertzog <hertzog@debian.org> | ||
| 208 | 10 | Standards-Version: 3.8.1 | ||
| 209 | 11 | Build-Depends: quilt, debhelper (>= 7) | ||
| 210 | 12 | Checksums-Sha1: | ||
| 211 | 13 | 296cb89762668772c3ae4799bd9ce7973f4154b2 171 sample1_1.0.orig-component1.tar.bz2 | ||
| 212 | 14 | ccb198be27c065880696cbeb6fb5aebf190c8544 157 sample1_1.0.orig-component2.tar.lzma | ||
| 213 | 15 | ea3c0c6ee23410a9c454d9b35681a80460343456 201 sample1_1.0.orig-component3.tar.gz | ||
| 214 | 16 | c62b2496d3ff0c64dcf6488c95e1d5cbc9f5f30d 201 sample1_1.0.orig.tar.gz | ||
| 215 | 17 | b3c92ffaabc2e3e95e5f74e1de94351fddcb065c 1327 sample1_1.0-1.debian.tar.gz | ||
| 216 | 18 | Checksums-Sha256: | ||
| 217 | 19 | a0871b73922a228ef8c1f4c1a7e1aef94d8a59d919218242655b7faae352d5c9 171 sample1_1.0.orig-component1.tar.bz2 | ||
| 218 | 20 | 39963ed90d70e918a6cae8d73c81f42dcb7f0dc283a4715f032a1e28b98d1a25 157 sample1_1.0.orig-component2.tar.lzma | ||
| 219 | 21 | dc174acae810faf5f9a679e042f480905622fe4a854510dad6e200dbe6381c8d 201 sample1_1.0.orig-component3.tar.gz | ||
| 220 | 22 | 9c0f0b42f6283614b204a47600cb9b0d8b19dfa4cb43c69e6a0a40f3d1064e0e 201 sample1_1.0.orig.tar.gz | ||
| 221 | 23 | 6864ceeb315ad7b6a59636a0251525cbd8e9b76c6e934f3746271d4ebce5da3c 1327 sample1_1.0-1.debian.tar.gz | ||
| 222 | 24 | Files: | ||
| 223 | 25 | 8a90ee77cf3c68c32bdb5c55d40d06f4 171 sample1_1.0.orig-component1.tar.bz2 | ||
| 224 | 26 | 3535d4766df47e531372802a0c2ddb82 157 sample1_1.0.orig-component2.tar.lzma | ||
| 225 | 27 | 673b42066e8e858208b5d0c7a006c5fb 201 sample1_1.0.orig-component3.tar.gz | ||
| 226 | 28 | 9047a93bf12a7c749045e9bc9ff9ee99 201 sample1_1.0.orig.tar.gz | ||
| 227 | 29 | 9404bc99cdb06876a988d4c2262c073e 1327 sample1_1.0-1.debian.tar.gz | ||
| 228 | 30 | |||
| 229 | 31 | -----BEGIN PGP SIGNATURE----- | ||
| 230 | 32 | Version: GnuPG v1.4.10 (GNU/Linux) | ||
| 231 | 33 | |||
| 232 | 34 | iEYEARECAAYFAktq7/kACgkQPa9Uoh7vUnahEQCgmRTupXtrHegzFOb5jQMOCSXS | ||
| 233 | 35 | /p4AoIroJtmjVryFpWqAABZTFGjeWo85 | ||
| 234 | 36 | =vwh8 | ||
| 235 | 37 | -----END PGP SIGNATURE----- | ||
| 236 | 0 | 38 | ||
| 237 | === added file 'lib/lp/soyuz/scripts/tests/sync_source_home/sample1_1.0.orig-component1.tar.bz2' | |||
| 238 | 1 | Binary files lib/lp/soyuz/scripts/tests/sync_source_home/sample1_1.0.orig-component1.tar.bz2 1970-01-01 00:00:00 +0000 and lib/lp/soyuz/scripts/tests/sync_source_home/sample1_1.0.orig-component1.tar.bz2 2010-02-11 16:55:27 +0000 differ | 39 | Binary files lib/lp/soyuz/scripts/tests/sync_source_home/sample1_1.0.orig-component1.tar.bz2 1970-01-01 00:00:00 +0000 and lib/lp/soyuz/scripts/tests/sync_source_home/sample1_1.0.orig-component1.tar.bz2 2010-02-11 16:55:27 +0000 differ |
| 239 | === added file 'lib/lp/soyuz/scripts/tests/sync_source_home/sample1_1.0.orig-component2.tar.lzma' | |||
| 240 | 2 | Binary files lib/lp/soyuz/scripts/tests/sync_source_home/sample1_1.0.orig-component2.tar.lzma 1970-01-01 00:00:00 +0000 and lib/lp/soyuz/scripts/tests/sync_source_home/sample1_1.0.orig-component2.tar.lzma 2010-02-11 16:55:27 +0000 differ | 40 | Binary files lib/lp/soyuz/scripts/tests/sync_source_home/sample1_1.0.orig-component2.tar.lzma 1970-01-01 00:00:00 +0000 and lib/lp/soyuz/scripts/tests/sync_source_home/sample1_1.0.orig-component2.tar.lzma 2010-02-11 16:55:27 +0000 differ |
| 241 | === added file 'lib/lp/soyuz/scripts/tests/sync_source_home/sample1_1.0.orig-component3.tar.gz' | |||
| 242 | 3 | Binary files lib/lp/soyuz/scripts/tests/sync_source_home/sample1_1.0.orig-component3.tar.gz 1970-01-01 00:00:00 +0000 and lib/lp/soyuz/scripts/tests/sync_source_home/sample1_1.0.orig-component3.tar.gz 2010-02-11 16:55:27 +0000 differ | 41 | Binary files lib/lp/soyuz/scripts/tests/sync_source_home/sample1_1.0.orig-component3.tar.gz 1970-01-01 00:00:00 +0000 and lib/lp/soyuz/scripts/tests/sync_source_home/sample1_1.0.orig-component3.tar.gz 2010-02-11 16:55:27 +0000 differ |
| 243 | === added file 'lib/lp/soyuz/scripts/tests/sync_source_home/sample1_1.0.orig.tar.gz' | |||
| 244 | 4 | Binary files lib/lp/soyuz/scripts/tests/sync_source_home/sample1_1.0.orig.tar.gz 1970-01-01 00:00:00 +0000 and lib/lp/soyuz/scripts/tests/sync_source_home/sample1_1.0.orig.tar.gz 2010-02-11 16:55:27 +0000 differ | 42 | Binary files lib/lp/soyuz/scripts/tests/sync_source_home/sample1_1.0.orig.tar.gz 1970-01-01 00:00:00 +0000 and lib/lp/soyuz/scripts/tests/sync_source_home/sample1_1.0.orig.tar.gz 2010-02-11 16:55:27 +0000 differ |
| 245 | === modified file 'lib/lp/soyuz/scripts/tests/test_sync_source.py' | |||
| 246 | --- lib/lp/soyuz/scripts/tests/test_sync_source.py 2009-06-25 04:06:00 +0000 | |||
| 247 | +++ lib/lp/soyuz/scripts/tests/test_sync_source.py 2010-02-11 16:55:27 +0000 | |||
| 248 | @@ -1,7 +1,7 @@ | |||
| 249 | 1 | # Copyright 2009 Canonical Ltd. This software is licensed under the | 1 | # Copyright 2009 Canonical Ltd. This software is licensed under the |
| 250 | 2 | # GNU Affero General Public License version 3 (see the file LICENSE). | 2 | # GNU Affero General Public License version 3 (see the file LICENSE). |
| 251 | 3 | 3 | ||
| 253 | 4 | """ChrootManager facilities tests.""" | 4 | """SyncSource facilities tests.""" |
| 254 | 5 | 5 | ||
| 255 | 6 | __metaclass__ = type | 6 | __metaclass__ = type |
| 256 | 7 | 7 | ||
| 257 | @@ -12,11 +12,15 @@ | |||
| 258 | 12 | import tempfile | 12 | import tempfile |
| 259 | 13 | from unittest import TestCase, TestLoader | 13 | from unittest import TestCase, TestLoader |
| 260 | 14 | 14 | ||
| 262 | 15 | from lp.archiveuploader.tagfiles import parse_tagfile | 15 | from zope.component import getUtility |
| 263 | 16 | |||
| 264 | 16 | from canonical.config import config | 17 | from canonical.config import config |
| 265 | 18 | from canonical.launchpad.scripts import BufferLogger | ||
| 266 | 17 | from canonical.librarian.ftests.harness import ( | 19 | from canonical.librarian.ftests.harness import ( |
| 267 | 18 | fillLibrarianFile, cleanupLibrarianFiles) | 20 | fillLibrarianFile, cleanupLibrarianFiles) |
| 268 | 19 | from canonical.testing import LaunchpadZopelessLayer | 21 | from canonical.testing import LaunchpadZopelessLayer |
| 269 | 22 | from lp.archiveuploader.tagfiles import parse_tagfile | ||
| 270 | 23 | from lp.registry.interfaces.distribution import IDistributionSet | ||
| 271 | 20 | from lp.soyuz.scripts.ftpmaster import ( | 24 | from lp.soyuz.scripts.ftpmaster import ( |
| 272 | 21 | SyncSource, SyncSourceError) | 25 | SyncSource, SyncSourceError) |
| 273 | 22 | 26 | ||
| 274 | @@ -37,7 +41,7 @@ | |||
| 275 | 37 | self._home = os.path.abspath('') | 41 | self._home = os.path.abspath('') |
| 276 | 38 | self._jail = tempfile.mkdtemp() | 42 | self._jail = tempfile.mkdtemp() |
| 277 | 39 | os.chdir(self._jail) | 43 | os.chdir(self._jail) |
| 279 | 40 | self.messages = [] | 44 | self.logger = BufferLogger() |
| 280 | 41 | self.downloads = [] | 45 | self.downloads = [] |
| 281 | 42 | 46 | ||
| 282 | 43 | def tearDown(self): | 47 | def tearDown(self): |
| 283 | @@ -54,9 +58,9 @@ | |||
| 284 | 54 | """Return a list of files present in jail.""" | 58 | """Return a list of files present in jail.""" |
| 285 | 55 | return os.listdir(self._jail) | 59 | return os.listdir(self._jail) |
| 286 | 56 | 60 | ||
| 290 | 57 | def local_debug(self, message): | 61 | def get_messages(self): |
| 291 | 58 | """Store debug messages for future inspection.""" | 62 | """Retrieve the messages sent using the logger.""" |
| 292 | 59 | self.messages.append(message) | 63 | return self.logger.buffer.getvalue().splitlines() |
| 293 | 60 | 64 | ||
| 294 | 61 | def local_downloader(self, url, filename): | 65 | def local_downloader(self, url, filename): |
| 295 | 62 | """Store download requests for future inspections.""" | 66 | """Store download requests for future inspections.""" |
| 296 | @@ -72,8 +76,9 @@ | |||
| 297 | 72 | them later. | 76 | them later. |
| 298 | 73 | """ | 77 | """ |
| 299 | 74 | sync_source = SyncSource( | 78 | sync_source = SyncSource( |
| 302 | 75 | files=files, origin=origin, debug=self.local_debug, | 79 | files=files, origin=origin, logger=self.logger, |
| 303 | 76 | downloader=self.local_downloader) | 80 | downloader=self.local_downloader, |
| 304 | 81 | todistro=getUtility(IDistributionSet)['ubuntu']) | ||
| 305 | 77 | return sync_source | 82 | return sync_source |
| 306 | 78 | 83 | ||
| 307 | 79 | def testInstantiate(self): | 84 | def testInstantiate(self): |
| 308 | @@ -86,8 +91,8 @@ | |||
| 309 | 86 | self.assertEqual(sync_source.files, files) | 91 | self.assertEqual(sync_source.files, files) |
| 310 | 87 | self.assertEqual(sync_source.origin, origin) | 92 | self.assertEqual(sync_source.origin, origin) |
| 311 | 88 | 93 | ||
| 314 | 89 | sync_source.debug('opa') | 94 | sync_source.logger.debug('opa') |
| 315 | 90 | self.assertEqual(self.messages, ['opa']) | 95 | self.assertEqual(self.get_messages(), ['DEBUG: opa']) |
| 316 | 91 | 96 | ||
| 317 | 92 | sync_source.downloader('somewhere', 'foo') | 97 | sync_source.downloader('somewhere', 'foo') |
| 318 | 93 | self.assertEqual(self.downloads, [('somewhere', 'foo')]) | 98 | self.assertEqual(self.downloads, [('somewhere', 'foo')]) |
| 319 | @@ -159,26 +164,26 @@ | |||
| 320 | 159 | test_file is skipped. | 164 | test_file is skipped. |
| 321 | 160 | """ | 165 | """ |
| 322 | 161 | files = { | 166 | files = { |
| 326 | 162 | 'foo.diff.gz': {'remote filename': 'xxx'}, | 167 | 'foo_0.1.diff.gz': {'remote filename': 'xxx'}, |
| 327 | 163 | 'foo.dsc': {'remote filename': 'yyy'}, | 168 | 'foo_0.1.dsc': {'remote filename': 'yyy'}, |
| 328 | 164 | 'foo.orig.gz': {'remote filename': 'zzz'}, | 169 | 'foo_0.1.orig.gz': {'remote filename': 'zzz'}, |
| 329 | 165 | } | 170 | } |
| 330 | 166 | origin = {'url': 'http://somewhere/'} | 171 | origin = {'url': 'http://somewhere/'} |
| 331 | 167 | 172 | ||
| 332 | 168 | sync_source = self._getSyncSource(files, origin) | 173 | sync_source = self._getSyncSource(files, origin) |
| 333 | 169 | 174 | ||
| 335 | 170 | test_file = open('foo.diff.gz', 'w') | 175 | test_file = open('foo_0.1.diff.gz', 'w') |
| 336 | 171 | test_file.write('nahhh') | 176 | test_file.write('nahhh') |
| 337 | 172 | test_file.close() | 177 | test_file.close() |
| 338 | 173 | 178 | ||
| 339 | 174 | dsc_filename = sync_source.fetchSyncFiles() | 179 | dsc_filename = sync_source.fetchSyncFiles() |
| 340 | 175 | 180 | ||
| 342 | 176 | self.assertEqual(dsc_filename, 'foo.dsc') | 181 | self.assertEqual(dsc_filename, 'foo_0.1.dsc') |
| 343 | 177 | 182 | ||
| 344 | 178 | self.assertEqual( | 183 | self.assertEqual( |
| 345 | 179 | self.downloads, | 184 | self.downloads, |
| 348 | 180 | [('http://somewhere/yyy', 'foo.dsc'), | 185 | [('http://somewhere/zzz', 'foo_0.1.orig.gz'), |
| 349 | 181 | ('http://somewhere/zzz', 'foo.orig.gz')]) | 186 | ('http://somewhere/yyy', 'foo_0.1.dsc')]) |
| 350 | 182 | 187 | ||
| 351 | 183 | for filename in files.keys(): | 188 | for filename in files.keys(): |
| 352 | 184 | self.assertTrue(os.path.exists(filename)) | 189 | self.assertTrue(os.path.exists(filename)) |
| 353 | @@ -196,13 +201,13 @@ | |||
| 354 | 196 | origin = {} | 201 | origin = {} |
| 355 | 197 | sync_source = self._getSyncSource(files, origin) | 202 | sync_source = self._getSyncSource(files, origin) |
| 356 | 198 | 203 | ||
| 358 | 199 | orig_filename = sync_source.fetchLibrarianFiles() | 204 | librarian_files = sync_source.fetchLibrarianFiles() |
| 359 | 200 | 205 | ||
| 361 | 201 | self.assertEqual(orig_filename, 'netapplet_1.0.0.orig.tar.gz') | 206 | self.assertEqual(librarian_files, ['netapplet_1.0.0.orig.tar.gz']) |
| 362 | 202 | self.assertEqual(self._listFiles(), ['netapplet_1.0.0.orig.tar.gz']) | 207 | self.assertEqual(self._listFiles(), ['netapplet_1.0.0.orig.tar.gz']) |
| 363 | 203 | self.assertEqual( | 208 | self.assertEqual( |
| 366 | 204 | self.messages, | 209 | self.get_messages(), |
| 367 | 205 | ['\tnetapplet_1.0.0.orig.tar.gz: already in distro ' | 210 | ['INFO: netapplet_1.0.0.orig.tar.gz: already in distro ' |
| 368 | 206 | '- downloading from librarian']) | 211 | '- downloading from librarian']) |
| 369 | 207 | 212 | ||
| 370 | 208 | def testFetchLibrarianFilesGotDuplicatedDSC(self): | 213 | def testFetchLibrarianFilesGotDuplicatedDSC(self): |
| 371 | @@ -224,8 +229,8 @@ | |||
| 372 | 224 | sync_source.fetchLibrarianFiles) | 229 | sync_source.fetchLibrarianFiles) |
| 373 | 225 | 230 | ||
| 374 | 226 | self.assertEqual( | 231 | self.assertEqual( |
| 377 | 227 | self.messages, | 232 | self.get_messages(), |
| 378 | 228 | ['\tfoobar-1.0.dsc: already in distro ' | 233 | ['INFO: foobar-1.0.dsc: already in distro ' |
| 379 | 229 | '- downloading from librarian']) | 234 | '- downloading from librarian']) |
| 380 | 230 | self.assertEqual(self._listFiles(), ['foobar-1.0.dsc']) | 235 | self.assertEqual(self._listFiles(), ['foobar-1.0.dsc']) |
| 381 | 231 | 236 | ||
| 382 | @@ -261,7 +266,7 @@ | |||
| 383 | 261 | stdout, stderr = process.communicate() | 266 | stdout, stderr = process.communicate() |
| 384 | 262 | return (process.returncode, stdout, stderr) | 267 | return (process.returncode, stdout, stderr) |
| 385 | 263 | 268 | ||
| 387 | 264 | def testSyncSourceRun(self): | 269 | def testSyncSourceRunV1(self): |
| 388 | 265 | """Try a simple sync-source.py run. | 270 | """Try a simple sync-source.py run. |
| 389 | 266 | 271 | ||
| 390 | 267 | It will run in a special tree prepared to cope with sync-source | 272 | It will run in a special tree prepared to cope with sync-source |
| 391 | @@ -288,15 +293,17 @@ | |||
| 392 | 288 | self.assertEqual( | 293 | self.assertEqual( |
| 393 | 289 | err.splitlines(), | 294 | err.splitlines(), |
| 394 | 290 | ['W: Could not find blacklist file on ' | 295 | ['W: Could not find blacklist file on ' |
| 396 | 291 | '/srv/launchpad.net/dak/sync-blacklist.txt']) | 296 | '/srv/launchpad.net/dak/sync-blacklist.txt', |
| 397 | 297 | 'INFO - <bar_1.0-1.diff.gz: cached>', | ||
| 398 | 298 | 'INFO - <bar_1.0.orig.tar.gz: cached>', | ||
| 399 | 299 | 'INFO - <bar_1.0-1.dsc: cached>', | ||
| 400 | 300 | ]) | ||
| 401 | 292 | self.assertEqual( | 301 | self.assertEqual( |
| 402 | 293 | out.splitlines(), | 302 | out.splitlines(), |
| 403 | 294 | ['Getting binaries for hoary...', | 303 | ['Getting binaries for hoary...', |
| 404 | 295 | '[Updating] bar (0 [Ubuntu] < 1.0-1 [Debian])', | 304 | '[Updating] bar (0 [Ubuntu] < 1.0-1 [Debian])', |
| 405 | 296 | ' * Trying to add bar...', | 305 | ' * Trying to add bar...', |
| 409 | 297 | ' - <bar_1.0-1.diff.gz: cached>', | 306 | ]) |
| 407 | 298 | ' - <bar_1.0.orig.tar.gz: cached>', | ||
| 408 | 299 | ' - <bar_1.0-1.dsc: cached>']) | ||
| 410 | 300 | 307 | ||
| 411 | 301 | expected_changesfile = 'bar_1.0-1_source.changes' | 308 | expected_changesfile = 'bar_1.0-1_source.changes' |
| 412 | 302 | self.assertTrue( | 309 | self.assertTrue( |
| 413 | @@ -331,6 +338,76 @@ | |||
| 414 | 331 | 338 | ||
| 415 | 332 | os.unlink(expected_changesfile) | 339 | os.unlink(expected_changesfile) |
| 416 | 333 | 340 | ||
| 417 | 341 | def testSyncSourceRunV3(self): | ||
| 418 | 342 | """Try a simple sync-source.py run with a version 3 source format | ||
| 419 | 343 | package. | ||
| 420 | 344 | |||
| 421 | 345 | It will run in a special tree prepared to cope with sync-source | ||
| 422 | 346 | requirements (see `setUp`). It contains a usable archive index | ||
| 423 | 347 | named as '$distribution_$suite_$component_Sources' and the | ||
| 424 | 348 | 'etherwake' source files. | ||
| 425 | 349 | |||
| 426 | 350 | Check that: | ||
| 427 | 351 | * return code is ZERO, | ||
| 428 | 352 | * check standard error and standard output, | ||
| 429 | 353 | * check if the expected changesfile was generated, | ||
| 430 | 354 | * parse and inspect the changesfile using the archiveuploader | ||
| 431 | 355 | component (the same approach adopted by Soyuz). | ||
| 432 | 356 | * delete the changesfile. | ||
| 433 | 357 | """ | ||
| 434 | 358 | returncode, out, err = self.runSyncSource( | ||
| 435 | 359 | extra_args=['-b', 'cprov', '-D', 'debian', '-C', 'main', | ||
| 436 | 360 | '-S', 'incoming', 'sample1']) | ||
| 437 | 361 | |||
| 438 | 362 | self.assertEqual( | ||
| 439 | 363 | 0, returncode, "\nScript Failed:%s\nStdout:\n%s\nStderr\n%s\n" | ||
| 440 | 364 | % (returncode, out, err)) | ||
| 441 | 365 | |||
| 442 | 366 | self.assertEqual( | ||
| 443 | 367 | err.splitlines(), | ||
| 444 | 368 | ['W: Could not find blacklist file on ' | ||
| 445 | 369 | '/srv/launchpad.net/dak/sync-blacklist.txt', | ||
| 446 | 370 | 'INFO - <sample1_1.0.orig-component3.tar.gz: cached>', | ||
| 447 | 371 | 'INFO - <sample1_1.0-1.dsc: cached>', | ||
| 448 | 372 | 'INFO - <sample1_1.0-1.debian.tar.gz: cached>', | ||
| 449 | 373 | 'INFO - <sample1_1.0.orig-component1.tar.bz2: cached>', | ||
| 450 | 374 | 'INFO - <sample1_1.0.orig-component2.tar.lzma: cached>', | ||
| 451 | 375 | 'INFO - <sample1_1.0.orig.tar.gz: cached>']) | ||
| 452 | 376 | self.assertEqual( | ||
| 453 | 377 | out.splitlines(), | ||
| 454 | 378 | ['Getting binaries for hoary...', | ||
| 455 | 379 | '[Updating] sample1 (0 [Ubuntu] < 1.0-1 [Debian])', | ||
| 456 | 380 | ' * Trying to add sample1...', | ||
| 457 | 381 | ]) | ||
| 458 | 382 | |||
| 459 | 383 | expected_changesfile = 'sample1_1.0-1_source.changes' | ||
| 460 | 384 | self.assertTrue( | ||
| 461 | 385 | os.path.exists(expected_changesfile), | ||
| 462 | 386 | "Couldn't find %s." % expected_changesfile) | ||
| 463 | 387 | |||
| 464 | 388 | # Parse the generated unsigned changesfile. | ||
| 465 | 389 | parsed_changes = parse_tagfile( | ||
| 466 | 390 | expected_changesfile, allow_unsigned=True) | ||
| 467 | 391 | |||
| 468 | 392 | # It refers to the right source/version. | ||
| 469 | 393 | self.assertEqual(parsed_changes['source'], 'sample1') | ||
| 470 | 394 | self.assertEqual(parsed_changes['version'], '1.0-1') | ||
| 471 | 395 | |||
| 472 | 396 | # It includes the correct 'origin' and 'target' information. | ||
| 473 | 397 | self.assertEqual(parsed_changes['origin'], 'Debian/incoming') | ||
| 474 | 398 | self.assertEqual(parsed_changes['distribution'], 'hoary') | ||
| 475 | 399 | |||
| 476 | 400 | # And finally, 'maintainer' role was preserved and 'changed-by' | ||
| 477 | 401 | # role was assigned as specified in the sync-source command-line. | ||
| 478 | 402 | self.assertEqual( | ||
| 479 | 403 | parsed_changes['maintainer'], | ||
| 480 | 404 | 'Raphael Hertzog <hertzog@debian.org>') | ||
| 481 | 405 | self.assertEqual( | ||
| 482 | 406 | parsed_changes['changed-by'], | ||
| 483 | 407 | 'Celso Providelo <celso.providelo@canonical.com>') | ||
| 484 | 408 | |||
| 485 | 409 | os.unlink(expected_changesfile) | ||
| 486 | 410 | |||
| 487 | 334 | 411 | ||
| 488 | 335 | def test_suite(): | 412 | def test_suite(): |
| 489 | 336 | return TestLoader().loadTestsFromName(__name__) | 413 | return TestLoader().loadTestsFromName(__name__) |
| 490 | 337 | 414 | ||
| 491 | === modified file 'scripts/ftpmaster-tools/dak_utils.py' | |||
| 492 | --- scripts/ftpmaster-tools/dak_utils.py 2009-11-18 02:58:23 +0000 | |||
| 493 | +++ scripts/ftpmaster-tools/dak_utils.py 2010-02-11 16:55:27 +0000 | |||
| 494 | @@ -34,156 +34,6 @@ | |||
| 495 | 34 | re_no_epoch = re.compile(r"^\d+\:") | 34 | re_no_epoch = re.compile(r"^\d+\:") |
| 496 | 35 | re_extract_src_version = re.compile(r"(\S+)\s*\((.*)\)") | 35 | re_extract_src_version = re.compile(r"(\S+)\s*\((.*)\)") |
| 497 | 36 | 36 | ||
| 498 | 37 | changes_parse_error_exc = "Can't parse line in .changes file" | ||
| 499 | 38 | invalid_dsc_format_exc = "Invalid .dsc file" | ||
| 500 | 39 | nk_format_exc = "Unknown Format: in .changes file"; | ||
| 501 | 40 | no_files_exc = "No Files: field in .dsc or .changes file."; | ||
| 502 | 41 | |||
| 503 | 42 | ################################################################################ | ||
| 504 | 43 | |||
| 505 | 44 | def parse_changes(filename, signing_rules=0): | ||
| 506 | 45 | """Parses a changes file and returns a dictionary where each field is a | ||
| 507 | 46 | key. The mandatory first argument is the filename of the .changes | ||
| 508 | 47 | file. | ||
| 509 | 48 | |||
| 510 | 49 | signing_rules is an optional argument: | ||
| 511 | 50 | |||
| 512 | 51 | o If signing_rules == -1, no signature is required. | ||
| 513 | 52 | o If signing_rules == 0 (the default), a signature is required. | ||
| 514 | 53 | o If signing_rules == 1, it turns on the same strict format checking | ||
| 515 | 54 | as dpkg-source. | ||
| 516 | 55 | |||
| 517 | 56 | The rules for (signing_rules == 1)-mode are: | ||
| 518 | 57 | |||
| 519 | 58 | o The PGP header consists of "-----BEGIN PGP SIGNED MESSAGE-----" | ||
| 520 | 59 | followed by any PGP header data and must end with a blank line. | ||
| 521 | 60 | |||
| 522 | 61 | o The data section must end with a blank line and must be followed by | ||
| 523 | 62 | "-----BEGIN PGP SIGNATURE-----". | ||
| 524 | 63 | """ | ||
| 525 | 64 | |||
| 526 | 65 | error = "" | ||
| 527 | 66 | changes = {} | ||
| 528 | 67 | |||
| 529 | 68 | changes_in = open(filename) | ||
| 530 | 69 | lines = changes_in.readlines() | ||
| 531 | 70 | |||
| 532 | 71 | if not lines: | ||
| 533 | 72 | raise changes_parse_error_exc, "[Empty changes file]" | ||
| 534 | 73 | |||
| 535 | 74 | # Reindex by line number so we can easily verify the format of | ||
| 536 | 75 | # .dsc files... | ||
| 537 | 76 | index = 0 | ||
| 538 | 77 | indexed_lines = {} | ||
| 539 | 78 | for line in lines: | ||
| 540 | 79 | index += 1 | ||
| 541 | 80 | indexed_lines[index] = line[:-1] | ||
| 542 | 81 | |||
| 543 | 82 | inside_signature = 0 | ||
| 544 | 83 | |||
| 545 | 84 | num_of_lines = len(indexed_lines.keys()) | ||
| 546 | 85 | index = 0 | ||
| 547 | 86 | first = -1 | ||
| 548 | 87 | while index < num_of_lines: | ||
| 549 | 88 | index += 1 | ||
| 550 | 89 | line = indexed_lines[index] | ||
| 551 | 90 | if line == "": | ||
| 552 | 91 | if signing_rules == 1: | ||
| 553 | 92 | index += 1 | ||
| 554 | 93 | if index > num_of_lines: | ||
| 555 | 94 | raise invalid_dsc_format_exc, index | ||
| 556 | 95 | line = indexed_lines[index] | ||
| 557 | 96 | if not line.startswith("-----BEGIN PGP SIGNATURE"): | ||
| 558 | 97 | raise invalid_dsc_format_exc, index | ||
| 559 | 98 | inside_signature = 0 | ||
| 560 | 99 | break | ||
| 561 | 100 | else: | ||
| 562 | 101 | continue | ||
| 563 | 102 | if line.startswith("-----BEGIN PGP SIGNATURE"): | ||
| 564 | 103 | break | ||
| 565 | 104 | if line.startswith("-----BEGIN PGP SIGNED MESSAGE"): | ||
| 566 | 105 | inside_signature = 1 | ||
| 567 | 106 | if signing_rules == 1: | ||
| 568 | 107 | while index < num_of_lines and line != "": | ||
| 569 | 108 | index += 1 | ||
| 570 | 109 | line = indexed_lines[index] | ||
| 571 | 110 | continue | ||
| 572 | 111 | # If we're not inside the signed data, don't process anything | ||
| 573 | 112 | if signing_rules >= 0 and not inside_signature: | ||
| 574 | 113 | continue | ||
| 575 | 114 | slf = re_single_line_field.match(line) | ||
| 576 | 115 | if slf: | ||
| 577 | 116 | field = slf.groups()[0].lower() | ||
| 578 | 117 | changes[field] = slf.groups()[1] | ||
| 579 | 118 | first = 1 | ||
| 580 | 119 | continue | ||
| 581 | 120 | if line == " .": | ||
| 582 | 121 | changes[field] += '\n' | ||
| 583 | 122 | continue | ||
| 584 | 123 | mlf = re_multi_line_field.match(line) | ||
| 585 | 124 | if mlf: | ||
| 586 | 125 | if first == -1: | ||
| 587 | 126 | raise changes_parse_error_exc, "'%s'\n [Multi-line field continuing on from nothing?]" % (line) | ||
| 588 | 127 | if first == 1 and changes[field] != "": | ||
| 589 | 128 | changes[field] += '\n' | ||
| 590 | 129 | first = 0 | ||
| 591 | 130 | changes[field] += mlf.groups()[0] + '\n' | ||
| 592 | 131 | continue | ||
| 593 | 132 | error += line | ||
| 594 | 133 | |||
| 595 | 134 | if signing_rules == 1 and inside_signature: | ||
| 596 | 135 | raise invalid_dsc_format_exc, index | ||
| 597 | 136 | |||
| 598 | 137 | changes_in.close() | ||
| 599 | 138 | changes["filecontents"] = "".join(lines) | ||
| 600 | 139 | |||
| 601 | 140 | if error: | ||
| 602 | 141 | raise changes_parse_error_exc, error | ||
| 603 | 142 | |||
| 604 | 143 | return changes | ||
| 605 | 144 | |||
| 606 | 145 | ################################################################################ | ||
| 607 | 146 | |||
| 608 | 147 | def build_file_list(changes, is_a_dsc=0): | ||
| 609 | 148 | files = {}; | ||
| 610 | 149 | |||
| 611 | 150 | # Make sure we have a Files: field to parse... | ||
| 612 | 151 | if not changes.has_key("files"): | ||
| 613 | 152 | raise no_files_exc; | ||
| 614 | 153 | |||
| 615 | 154 | # Make sure we recognise the format of the Files: field | ||
| 616 | 155 | format = changes.get("format", ""); | ||
| 617 | 156 | if format != "": | ||
| 618 | 157 | format = float(format.split()[0]); | ||
| 619 | 158 | if not is_a_dsc and (format < 1.5 or format > 2.0): | ||
| 620 | 159 | raise nk_format_exc, format; | ||
| 621 | 160 | |||
| 622 | 161 | # Parse each entry/line: | ||
| 623 | 162 | for i in changes["files"].split('\n'): | ||
| 624 | 163 | if not i: | ||
| 625 | 164 | break; | ||
| 626 | 165 | s = i.split(); | ||
| 627 | 166 | section = priority = ""; | ||
| 628 | 167 | try: | ||
| 629 | 168 | if is_a_dsc: | ||
| 630 | 169 | (md5, size, name) = s; | ||
| 631 | 170 | else: | ||
| 632 | 171 | (md5, size, section, priority, name) = s; | ||
| 633 | 172 | except ValueError: | ||
| 634 | 173 | raise changes_parse_error_exc, i; | ||
| 635 | 174 | |||
| 636 | 175 | if section == "": | ||
| 637 | 176 | section = "-"; | ||
| 638 | 177 | if priority == "": | ||
| 639 | 178 | priority = "-"; | ||
| 640 | 179 | |||
| 641 | 180 | (section, component) = extract_component_from_section(section); | ||
| 642 | 181 | |||
| 643 | 182 | files[name] = Dict(md5sum=md5, size=size, section=section, | ||
| 644 | 183 | priority=priority, component=component); | ||
| 645 | 184 | |||
| 646 | 185 | return files | ||
| 647 | 186 | |||
| 648 | 187 | ################################################################################ | 37 | ################################################################################ |
| 649 | 188 | 38 | ||
| 650 | 189 | def fubar(msg, exit_code=1): | 39 | def fubar(msg, exit_code=1): |
| 651 | @@ -309,9 +159,9 @@ | |||
| 652 | 309 | ################################################################################ | 159 | ################################################################################ |
| 653 | 310 | 160 | ||
| 654 | 311 | def join_with_commas_and(list): | 161 | def join_with_commas_and(list): |
| 658 | 312 | if len(list) == 0: return "nothing"; | 162 | if len(list) == 0: return "nothing"; |
| 659 | 313 | if len(list) == 1: return list[0]; | 163 | if len(list) == 1: return list[0]; |
| 660 | 314 | return ", ".join(list[:-1]) + " and " + list[-1]; | 164 | return ", ".join(list[:-1]) + " and " + list[-1]; |
| 661 | 315 | 165 | ||
| 662 | 316 | ################################################################################ | 166 | ################################################################################ |
| 663 | 317 | 167 | ||
| 664 | 318 | 168 | ||
| 665 | === modified file 'scripts/ftpmaster-tools/sync-source.py' | |||
| 666 | --- scripts/ftpmaster-tools/sync-source.py 2010-02-02 22:34:39 +0000 | |||
| 667 | +++ scripts/ftpmaster-tools/sync-source.py 2010-02-11 16:55:27 +0000 | |||
| 668 | @@ -18,6 +18,7 @@ | |||
| 669 | 18 | 18 | ||
| 670 | 19 | import apt_pkg | 19 | import apt_pkg |
| 671 | 20 | import commands | 20 | import commands |
| 672 | 21 | from debian_bundle.deb822 import Dsc | ||
| 673 | 21 | import errno | 22 | import errno |
| 674 | 22 | import optparse | 23 | import optparse |
| 675 | 23 | import os | 24 | import os |
| 676 | @@ -25,7 +26,6 @@ | |||
| 677 | 25 | import shutil | 26 | import shutil |
| 678 | 26 | import stat | 27 | import stat |
| 679 | 27 | import string | 28 | import string |
| 680 | 28 | import sys | ||
| 681 | 29 | import tempfile | 29 | import tempfile |
| 682 | 30 | import time | 30 | import time |
| 683 | 31 | import urllib | 31 | import urllib |
| 684 | @@ -44,6 +44,7 @@ | |||
| 685 | 44 | from canonical.librarian.client import LibrarianClient | 44 | from canonical.librarian.client import LibrarianClient |
| 686 | 45 | from canonical.lp import initZopeless | 45 | from canonical.lp import initZopeless |
| 687 | 46 | from lp.registry.interfaces.pocket import PackagePublishingPocket | 46 | from lp.registry.interfaces.pocket import PackagePublishingPocket |
| 688 | 47 | from lp.soyuz.scripts.ftpmaster import SyncSource, SyncSourceError | ||
| 689 | 47 | 48 | ||
| 690 | 48 | 49 | ||
| 691 | 49 | reject_message = "" | 50 | reject_message = "" |
| 692 | @@ -130,7 +131,7 @@ | |||
| 693 | 130 | 131 | ||
| 694 | 131 | def generate_changes(dsc, dsc_files, suite, changelog, urgency, closes, | 132 | def generate_changes(dsc, dsc_files, suite, changelog, urgency, closes, |
| 695 | 132 | lp_closes, section, priority, description, | 133 | lp_closes, section, priority, description, |
| 697 | 133 | have_orig_tar_gz, requested_by, origin): | 134 | files_from_librarian, requested_by, origin): |
| 698 | 134 | """Generate a .changes as a string""" | 135 | """Generate a .changes as a string""" |
| 699 | 135 | 136 | ||
| 700 | 136 | # XXX cprov 2007-07-03: | 137 | # XXX cprov 2007-07-03: |
| 701 | @@ -164,7 +165,7 @@ | |||
| 702 | 164 | changes += changelog | 165 | changes += changelog |
| 703 | 165 | changes += "Files: \n" | 166 | changes += "Files: \n" |
| 704 | 166 | for filename in dsc_files: | 167 | for filename in dsc_files: |
| 706 | 167 | if filename.endswith(".orig.tar.gz") and have_orig_tar_gz: | 168 | if filename in files_from_librarian: |
| 707 | 168 | continue | 169 | continue |
| 708 | 169 | changes += " %s %s %s %s %s\n" % (dsc_files[filename]["md5sum"], | 170 | changes += " %s %s %s %s %s\n" % (dsc_files[filename]["md5sum"], |
| 709 | 170 | dsc_files[filename]["size"], | 171 | dsc_files[filename]["size"], |
| 710 | @@ -361,13 +362,91 @@ | |||
| 711 | 361 | source, source_component, binary, current_version, | 362 | source, source_component, binary, current_version, |
| 712 | 362 | current_component) | 363 | current_component) |
| 713 | 363 | 364 | ||
| 714 | 365 | def split_gpg_and_payload(sequence): | ||
| 715 | 366 | """Return a (gpg_pre, payload, gpg_post) tuple | ||
| 716 | 367 | |||
| 717 | 368 | Each element of the returned tuple is a list of lines (with trailing | ||
| 718 | 369 | whitespace stripped). | ||
| 719 | 370 | """ | ||
| 720 | 371 | # XXX JRV 20100211: Copied from deb822.py in python-debian. When | ||
| 721 | 372 | # Launchpad switches to Lucid this copy should be removed. | ||
| 722 | 373 | # bug=520508 | ||
| 723 | 374 | |||
| 724 | 375 | gpg_pre_lines = [] | ||
| 725 | 376 | lines = [] | ||
| 726 | 377 | gpg_post_lines = [] | ||
| 727 | 378 | state = 'SAFE' | ||
| 728 | 379 | gpgre = re.compile(r'^-----(?P<action>BEGIN|END) PGP (?P<what>[^-]+)-----$') | ||
| 729 | 380 | blank_line = re.compile('^$') | ||
| 730 | 381 | first_line = True | ||
| 731 | 382 | |||
| 732 | 383 | for line in sequence: | ||
| 733 | 384 | line = line.strip('\r\n') | ||
| 734 | 385 | |||
| 735 | 386 | # skip initial blank lines, if any | ||
| 736 | 387 | if first_line: | ||
| 737 | 388 | if blank_line.match(line): | ||
| 738 | 389 | continue | ||
| 739 | 390 | else: | ||
| 740 | 391 | first_line = False | ||
| 741 | 392 | |||
| 742 | 393 | m = gpgre.match(line) | ||
| 743 | 394 | |||
| 744 | 395 | if not m: | ||
| 745 | 396 | if state == 'SAFE': | ||
| 746 | 397 | if not blank_line.match(line): | ||
| 747 | 398 | lines.append(line) | ||
| 748 | 399 | else: | ||
| 749 | 400 | if not gpg_pre_lines: | ||
| 750 | 401 | # There's no gpg signature, so we should stop at | ||
| 751 | 402 | # this blank line | ||
| 752 | 403 | break | ||
| 753 | 404 | elif state == 'SIGNED MESSAGE': | ||
| 754 | 405 | if blank_line.match(line): | ||
| 755 | 406 | state = 'SAFE' | ||
| 756 | 407 | else: | ||
| 757 | 408 | gpg_pre_lines.append(line) | ||
| 758 | 409 | elif state == 'SIGNATURE': | ||
| 759 | 410 | gpg_post_lines.append(line) | ||
| 760 | 411 | else: | ||
| 761 | 412 | if m.group('action') == 'BEGIN': | ||
| 762 | 413 | state = m.group('what') | ||
| 763 | 414 | elif m.group('action') == 'END': | ||
| 764 | 415 | gpg_post_lines.append(line) | ||
| 765 | 416 | break | ||
| 766 | 417 | if not blank_line.match(line): | ||
| 767 | 418 | if not lines: | ||
| 768 | 419 | gpg_pre_lines.append(line) | ||
| 769 | 420 | else: | ||
| 770 | 421 | gpg_post_lines.append(line) | ||
| 771 | 422 | |||
| 772 | 423 | if len(lines): | ||
| 773 | 424 | return (gpg_pre_lines, lines, gpg_post_lines) | ||
| 774 | 425 | else: | ||
| 775 | 426 | raise EOFError('only blank lines found in input') | ||
| 776 | 427 | |||
| 777 | 364 | 428 | ||
| 778 | 365 | def import_dsc(dsc_filename, suite, previous_version, signing_rules, | 429 | def import_dsc(dsc_filename, suite, previous_version, signing_rules, |
| 780 | 366 | have_orig_tar_gz, requested_by, origin, current_sources, | 430 | files_from_librarian, requested_by, origin, current_sources, |
| 781 | 367 | current_binaries): | 431 | current_binaries): |
| 785 | 368 | dsc = dak_utils.parse_changes(dsc_filename, signing_rules) | 432 | dsc_file = open(dsc_filename, 'r') |
| 786 | 369 | dsc_files = dak_utils.build_file_list(dsc, is_a_dsc=1)\ | 433 | dsc = Dsc(dsc_file) |
| 787 | 370 | 434 | ||
| 788 | 435 | if signing_rules.startswith("must be signed"): | ||
| 789 | 436 | dsc_file.seek(0) | ||
| 790 | 437 | # XXX JRV 20100211: When Launchpad starts depending on Lucid, | ||
| 791 | 438 | # use dsc.split_gpg_and_payload() instead. | ||
| 792 | 439 | # bug=520508 | ||
| 793 | 440 | (gpg_pre, payload, gpg_post) = split_gpg_and_payload(dsc_file) | ||
| 794 | 441 | if gpg_pre == [] and gpg_post == []: | ||
| 795 | 442 | dak_utils.fubar("signature required for %s but not present" | ||
| 796 | 443 | % dsc_filename) | ||
| 797 | 444 | if signing_rules == "must be signed and valid": | ||
| 798 | 445 | if (gpg_pre[0] != "-----BEGIN PGP SIGNED MESSAGE-----" or | ||
| 799 | 446 | gpg_post[0] != "-----BEGIN PGP SIGNATURE-----"): | ||
| 800 | 447 | dak_utils.fubar("signature for %s invalid %r %r" % (dsc_filename, gpg_pre, gpg_post)) | ||
| 801 | 448 | |||
| 802 | 449 | dsc_files = dict((entry['name'], entry) for entry in dsc['files']) | ||
| 803 | 371 | check_dsc(dsc, current_sources, current_binaries) | 450 | check_dsc(dsc, current_sources, current_binaries) |
| 804 | 372 | 451 | ||
| 805 | 373 | # Add the .dsc itself to dsc_files so it's listed in the Files: field | 452 | # Add the .dsc itself to dsc_files so it's listed in the Files: field |
| 806 | @@ -400,7 +479,7 @@ | |||
| 807 | 400 | 479 | ||
| 808 | 401 | changes = generate_changes( | 480 | changes = generate_changes( |
| 809 | 402 | dsc, dsc_files, suite, changelog, urgency, closes, lp_closes, | 481 | dsc, dsc_files, suite, changelog, urgency, closes, lp_closes, |
| 811 | 403 | section, priority, description, have_orig_tar_gz, requested_by, | 482 | section, priority, description, files_from_librarian, requested_by, |
| 812 | 404 | origin) | 483 | origin) |
| 813 | 405 | 484 | ||
| 814 | 406 | # XXX cprov 2007-07-03: Soyuz wants an unsigned changes | 485 | # XXX cprov 2007-07-03: Soyuz wants an unsigned changes |
| 815 | @@ -565,86 +644,21 @@ | |||
| 816 | 565 | if not Sources.has_key(pkg): | 644 | if not Sources.has_key(pkg): |
| 817 | 566 | dak_utils.fubar("%s doesn't exist in the Sources file." % (pkg)) | 645 | dak_utils.fubar("%s doesn't exist in the Sources file." % (pkg)) |
| 818 | 567 | 646 | ||
| 899 | 568 | have_orig_tar_gz = False | 647 | syncsource = SyncSource(Sources[pkg]["files"], origin, Log, |
| 900 | 569 | 648 | urllib.urlretrieve, Options.todistro) | |
| 901 | 570 | # Fetch the source | 649 | try: |
| 902 | 571 | files = Sources[pkg]["files"] | 650 | files_from_librarian = syncsource.fetchLibrarianFiles() |
| 903 | 572 | for filename in files: | 651 | dsc_filename = syncsource.fetchSyncFiles() |
| 904 | 573 | # First see if we can find the source in the librarian | 652 | syncsource.checkDownloadedFiles() |
| 905 | 574 | archive_ids = [a.id for a in Options.todistro.all_distro_archives] | 653 | except SyncSourceError, e: |
| 906 | 575 | query = """ | 654 | dak_utils.fubar("Fetching files failed: %s" % (str(e),)) |
| 907 | 576 | SELECT DISTINCT ON (LibraryFileContent.sha1, LibraryFileContent.filesize) | 655 | |
| 908 | 577 | LibraryFileAlias.id | 656 | if dsc_filename is None: |
| 909 | 578 | FROM SourcePackageFilePublishing, LibraryFileAlias, LibraryFileContent | 657 | dak_utils.fubar("No dsc filename in %r" % Sources[pkg]["files"].keys()) |
| 910 | 579 | WHERE | 658 | |
| 911 | 580 | LibraryFileAlias.id = SourcePackageFilePublishing.libraryfilealias AND | 659 | import_dsc(os.path.abspath(dsc_filename), suite, previous_version, |
| 912 | 581 | LibraryFileContent.id = LibraryFileAlias.content AND | 660 | origin["dsc"], files_from_librarian, requested_by, origin, |
| 913 | 582 | SourcePackageFilePublishing.libraryfilealiasfilename = %s AND | 661 | current_sources, current_binaries) |
| 834 | 583 | SourcePackageFilePublishing.archive IN %s | ||
| 835 | 584 | """ % sqlvalues(filename, archive_ids) | ||
| 836 | 585 | cur = cursor() | ||
| 837 | 586 | cur.execute(query) | ||
| 838 | 587 | results = cur.fetchall() | ||
| 839 | 588 | if results: | ||
| 840 | 589 | if not filename.endswith("orig.tar.gz"): | ||
| 841 | 590 | dak_utils.fubar( | ||
| 842 | 591 | "%s (from %s) is in the DB but isn't an orig.tar.gz. " | ||
| 843 | 592 | "(Probably published in an older release)" | ||
| 844 | 593 | % (filename, pkg)) | ||
| 845 | 594 | if len(results) > 1: | ||
| 846 | 595 | dak_utils.fubar( | ||
| 847 | 596 | "%s (from %s) returns multiple IDs (%s) for " | ||
| 848 | 597 | "orig.tar.gz. Help?" % (filename, pkg, results)) | ||
| 849 | 598 | have_orig_tar_gz = filename | ||
| 850 | 599 | print (" - <%s: already in distro - downloading from librarian>" | ||
| 851 | 600 | % (filename)) | ||
| 852 | 601 | output_file = open(filename, 'w') | ||
| 853 | 602 | librarian_input = Library.getFileByAlias(results[0][0]) | ||
| 854 | 603 | output_file.write(librarian_input.read()) | ||
| 855 | 604 | output_file.close() | ||
| 856 | 605 | continue | ||
| 857 | 606 | |||
| 858 | 607 | # Download the file | ||
| 859 | 608 | download_f = ( | ||
| 860 | 609 | "%s%s" % (origin["url"], files[filename]["remote filename"])) | ||
| 861 | 610 | if not os.path.exists(filename): | ||
| 862 | 611 | print " - <%s: downloading from %s>" % (filename, origin["url"]) | ||
| 863 | 612 | sys.stdout.flush() | ||
| 864 | 613 | urllib.urlretrieve(download_f, filename) | ||
| 865 | 614 | else: | ||
| 866 | 615 | print " - <%s: cached>" % (filename) | ||
| 867 | 616 | |||
| 868 | 617 | # Check md5sum and size match Source | ||
| 869 | 618 | actual_md5sum = md5sum_file(filename) | ||
| 870 | 619 | expected_md5sum = files[filename]["md5sum"] | ||
| 871 | 620 | if actual_md5sum != expected_md5sum: | ||
| 872 | 621 | dak_utils.fubar( | ||
| 873 | 622 | "%s: md5sum check failed (%s [actual] vs. %s [expected])." | ||
| 874 | 623 | % (filename, actual_md5sum, expected_md5sum)) | ||
| 875 | 624 | actual_size = os.stat(filename)[stat.ST_SIZE] | ||
| 876 | 625 | expected_size = int(files[filename]["size"]) | ||
| 877 | 626 | if actual_size != expected_size: | ||
| 878 | 627 | dak_utils.fubar( | ||
| 879 | 628 | "%s: size mismatch (%s [actual] vs. %s [expected])." | ||
| 880 | 629 | % (filename, actual_size, expected_size)) | ||
| 881 | 630 | |||
| 882 | 631 | # Remember the name of the .dsc file | ||
| 883 | 632 | if filename.endswith(".dsc"): | ||
| 884 | 633 | dsc_filename = os.path.abspath(filename) | ||
| 885 | 634 | |||
| 886 | 635 | if origin["dsc"] == "must be signed and valid": | ||
| 887 | 636 | signing_rules = 1 | ||
| 888 | 637 | elif origin["dsc"] == "must be signed": | ||
| 889 | 638 | signing_rules = 0 | ||
| 890 | 639 | else: | ||
| 891 | 640 | signing_rules = -1 | ||
| 892 | 641 | |||
| 893 | 642 | import_dsc(dsc_filename, suite, previous_version, signing_rules, | ||
| 894 | 643 | have_orig_tar_gz, requested_by, origin, current_sources, | ||
| 895 | 644 | current_binaries) | ||
| 896 | 645 | |||
| 897 | 646 | if have_orig_tar_gz: | ||
| 898 | 647 | os.unlink(have_orig_tar_gz) | ||
| 914 | 648 | 662 | ||
| 915 | 649 | 663 | ||
| 916 | 650 | def do_diff(Sources, Suite, origin, arguments, current_binaries): | 664 | def do_diff(Sources, Suite, origin, arguments, current_binaries): |
Fixes support for syncing packages in the the version 3 source format (bug 225151).
This patch adds support for .orig.tar.bz2 files as well as multiple component orig tarballs in the script that syncs source packages from Debian. These new file formats are used by the new Debian source package format, v3.0.
This branch also refactors sync-source.py to use code from lp.soyuz.