Merge lp:~cjwatson/turnip/auto-create-repository into lp:turnip

Proposed by Colin Watson
Status: Rejected
Rejected by: Colin Watson
Proposed branch: lp:~cjwatson/turnip/auto-create-repository
Merge into: lp:turnip
Diff against target: 95 lines (+22/-10)
3 files modified
turnip/api.py (+5/-9)
turnip/helpers.py (+11/-0)
turnip/pack/git.py (+6/-1)
To merge this branch: bzr merge lp:~cjwatson/turnip/auto-create-repository
Reviewer Review Type Date Requested Status
Canonical Launchpad Branches Pending
Review via email: mp+249058@code.launchpad.net

Commit message

Attempt to automatically create non-existent repositories on push.

Description of the change

Attempt to automatically create non-existent repositories on push.

This relies on a new createRepository XML-RPC call. I haven't planned to implement this in turnipcake, but I have a working (though not yet pushed anywhere) implementation in Launchpad.

To post a comment you must log in.
Revision history for this message
William Grant (wgrant) wrote :

I was thinking that this could instead be done by Launchpad inside translatePath if it knows it hasn't created it yet. What do you think about that?

93. By Colin Watson

Revert previous commit.

94. By Colin Watson

Ensure that a repository exists on disk before receiving pack data into it.

Revision history for this message
Colin Watson (cjwatson) wrote :

When trying to implement William's suggestion, I found that indeed it's quite reasonable for Launchpad to take care of creating a GitRepository row when asked to translate-for-writing a well-formed path it doesn't know about yet, but it's still useful for turnip to deal with the "git init" bit when trying to receive into a repository path that doesn't yet exist on disk. This makes the interface between Launchpad and turnip much simpler when creating repositories.

I've amended this branch with this new approach.

Revision history for this message
William Grant (wgrant) wrote :

On 14/02/15 05:28, Colin Watson wrote:
> When trying to implement William's suggestion, I found that indeed it's quite reasonable for Launchpad to take care of creating a GitRepository row when asked to translate-for-writing a well-formed path it doesn't know about yet, but it's still useful for turnip to deal with the "git init" bit when trying to receive into a repository path that doesn't yet exist on disk. This makes the interface between Launchpad and turnip much simpler when creating repositories.
>
> I've amended this branch with this new approach.

Will that still be useful when 99% of created repositories want to be
forks of existing ones, rather than fresh inits?

Revision history for this message
Colin Watson (cjwatson) wrote :

It probably won't be *useful* in that case, but also not harmful; I mean, if we have to do something first to fork the repository, then we have to do that no matter what, right?

Revision history for this message
William Grant (wgrant) wrote :

On 14/02/15 20:34, Colin Watson wrote:
> It probably won't be *useful* in that case, but also not harmful; I
> mean, if we have to do something first to fork the repository, then
> we have to do that no matter what, right?

If we run into this case on production it must indicate a bug, and
rather than reporting a problem it will just silently do the wrong thing.

In what case would this be useful in a real environment? Also remember
that we currently have no local state except the repository, but that
will eventually change, at which point implicit creation on ENOENT
becomes less desirable.

I would have Launchpad's translatePath create the LP GitRepository and
then call into turnip to create the repo.

Revision history for this message
Colin Watson (cjwatson) wrote :

I think I was misled by your comment on https://code.launchpad.net/~cjwatson/launchpad/git-namespace/+merge/248995 where you said "We might also be able to get away with doing this on the first repository access" into thinking that we could do part of it on the first access *in turnip*, and didn't think properly about the state requirements. Fair enough; I'll withdraw this MP, and include GitHostingClient again in Launchpad when I get far enough up the stack to have the XML-RPC service code.

Revision history for this message
William Grant (wgrant) wrote :

Ah, sorry, that was ambiguous indeed. I was meaning something sort of like the PPA key generation situation: we may assume that some reasonable subset of created repositories will never be accessed, so it may make sense to lazily create the turnip side of things. If we were to do that, LP's translatePath would notice that it has a GitRepository row but doesn't exist in turnip, and make the same call into turnip as it would if the GitRepository didn't exist. But Git repositories are cheaper than OpenPGP keys, so it may not be sensible to delay that except for app performance reasons.

Unmerged revisions

94. By Colin Watson

Ensure that a repository exists on disk before receiving pack data into it.

93. By Colin Watson

Revert previous commit.

92. By Colin Watson

Attempt to automatically create non-existent repositories on push.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'turnip/api.py'
--- turnip/api.py 2015-02-03 16:59:42 +0000
+++ turnip/api.py 2015-02-13 18:24:05 +0000
@@ -4,17 +4,17 @@
4 unicode_literals,4 unicode_literals,
5 )5 )
66
7import os
8
9from twisted.internet import defer7from twisted.internet import defer
10from twisted.internet.utils import getProcessValue
11from twisted.web import (8from twisted.web import (
12 resource,9 resource,
13 server,10 server,
14 static,11 static,
15 )12 )
1613
17from turnip.helpers import compose_path14from turnip.helpers import (
15 compose_path,
16 ensure_repository_exists,
17 )
1818
1919
20class TurnipAPIResource(resource.Resource):20class TurnipAPIResource(resource.Resource):
@@ -44,11 +44,7 @@
44 @defer.inlineCallbacks44 @defer.inlineCallbacks
45 def createRepo(self, request, raw_path):45 def createRepo(self, request, raw_path):
46 repo_path = compose_path(self.root, raw_path)46 repo_path = compose_path(self.root, raw_path)
47 if os.path.exists(repo_path):47 yield ensure_repository_exists(repo_path)
48 raise Exception("Repository '%s' already exists" % repo_path)
49 ret = yield getProcessValue('git', ('init', '--bare', repo_path))
50 if ret != 0:
51 raise Exception("'git init' failed")
52 request.write(b'OK')48 request.write(b'OK')
53 request.finish()49 request.finish()
5450
5551
=== modified file 'turnip/helpers.py'
--- turnip/helpers.py 2015-01-22 06:14:32 +0000
+++ turnip/helpers.py 2015-02-13 18:24:05 +0000
@@ -6,6 +6,9 @@
66
7import os.path7import os.path
88
9from twisted.internet import defer
10from twisted.internet.utils import getProcessValue
11
912
10def compose_path(root, path):13def compose_path(root, path):
11 # Construct the full path, stripping any leading slashes so we14 # Construct the full path, stripping any leading slashes so we
@@ -15,3 +18,11 @@
15 if not full_path.startswith(os.path.abspath(root)):18 if not full_path.startswith(os.path.abspath(root)):
16 raise ValueError('Path not contained within root')19 raise ValueError('Path not contained within root')
17 return full_path20 return full_path
21
22
23@defer.inlineCallbacks
24def ensure_repository_exists(repo_path):
25 if not os.path.exists(repo_path):
26 ret = yield getProcessValue('git', ('init', '--bare', repo_path))
27 if ret != 0:
28 raise Exception("'git init' failed")
1829
=== modified file 'turnip/pack/git.py'
--- turnip/pack/git.py 2015-02-09 10:23:55 +0000
+++ turnip/pack/git.py 2015-02-13 18:24:05 +0000
@@ -18,7 +18,10 @@
18 from twisted.web import xmlrpc18 from twisted.web import xmlrpc
19from zope.interface import implementer19from zope.interface import implementer
2020
21from turnip.helpers import compose_path21from turnip.helpers import (
22 compose_path,
23 ensure_repository_exists,
24 )
22from turnip.pack.helpers import (25from turnip.pack.helpers import (
23 decode_packet,26 decode_packet,
24 decode_request,27 decode_request,
@@ -271,12 +274,14 @@
271 Invokes the reference C Git implementation.274 Invokes the reference C Git implementation.
272 """275 """
273276
277 @defer.inlineCallbacks
274 def requestReceived(self, command, raw_pathname, params):278 def requestReceived(self, command, raw_pathname, params):
275 path = compose_path(self.factory.root, raw_pathname)279 path = compose_path(self.factory.root, raw_pathname)
276 if command == b'git-upload-pack':280 if command == b'git-upload-pack':
277 subcmd = b'upload-pack'281 subcmd = b'upload-pack'
278 elif command == b'git-receive-pack':282 elif command == b'git-receive-pack':
279 subcmd = b'receive-pack'283 subcmd = b'receive-pack'
284 yield ensure_repository_exists(path)
280 else:285 else:
281 self.die(b'Unsupport command in request')286 self.die(b'Unsupport command in request')
282 return287 return

Subscribers

People subscribed via source and target branches

to all changes: