Forums : Technical Issue Help

Hi,

we've waited along time before we finally moved our project from a closed CVS server to an open SVN server, so we could finally add our code to ohloh.

Ahlas, the import seems to fail.

http://www.ohloh.net/projects/3330/enlistments

What could be the problem? I've succesfully checked out the repository using cmdline svn and tortoisesvn.

Ivo Jansch about 16 years ago

Hi ivo,

It looks like there is some sort of problem getting the log from this server. Does the following work for you?

$ svn log --stop-on-copy --verbose -r1:HEAD http://svn.ibuildings.nl/atk/atk/trunk > ~/log
svn: REPORT request failed on '/atk/!svn/bc/5759/atk/trunk'
svn: REPORT of '/atk/!svn/bc/5759/atk/trunk': 200 OK (http://svn.ibuildings.nl)

I tried 3 or 4 times, and each time the log would fail after revision 4349. I spent some time working on other things, then tried again. Now the log fails consistently after revision 1874.

I have no idea what could be causing this problem, but it looks like a server-side issue. Any ideas?

Robin

Robin Luckey about 16 years ago

Hmm, indeed, that command fails here as well. I'll have our sysadmin look at it and post the result here. It's a conversion from a CVS repository, maybe it has something to do with that.

Ivo Jansch about 16 years ago

Hi Robin,

we've checked the repository. It appears some non-ascii chars were in the log messages of the CVS repository. cvs2svn converted them, but the log command chokes on them. We've edited the log entries and removed the characters. I was now able to run the svn log command succesfully. Could you please try again?

Ivo Jansch about 16 years ago

Hi ivo,

Yes, the log seems to be OK now. I've started a fresh download. Let me know if anything else unusual happens!

Thanks,
Robin

Robin Luckey about 16 years ago

I saw it running for a while, but now it's set to 'failed' again. Can you see what went wrong this time?

Ivo Jansch about 16 years ago

Hi ivo,

We had a checkout failure -- often this because of a server disconnect. Any chance we are bumping into some bandwidth limits or overstressing the server? After a break, I rescheduled the job. I'll keep an eye on it.

Thanks,
Robin

Robin Luckey about 16 years ago

Hmm, no, this runs on a server that is part of a cluster that runs most of our websites too. It's being used heavily throughout the day as our entire development team (40+) is working with that SVN server so I guess one ohloh checkout shouldn't really be a problem. :) I'll have our sysadmin look at it.

Ivo Jansch about 16 years ago

Hi ivo,

After a few failed retries I did some digging and found that there is an error checking out one of the revisions:

$ svn checkout -r 1875 'http://svn.ibuildings.nl/atk/atk/trunk@1875' --ignore-externals

... many files checked out and then ...

A trunk/doc
svn: REPORT request failed on '/atk/!svn/vcc/default'
svn: REPORT of '/atk/!svn/vcc/default': 200 OK (http://svn.ibuildings.nl)

I was able to check out revision 1874, so I believe this might be a data problem with this particular revision.

Robin Luckey about 16 years ago

We've checked and there appears to be a checksum error in the file. We also checked other files for the same type of errors (using 'svnadmin verify') and corrected all of them. I wouldn't want to waste your time by trial and error, so maybe you can give me some information on the commands that your scripts will run, so we can test them before you try again?

Ivo Jansch about 16 years ago

Hi ivo,

Thanks for the help. Our Subversion scripts are brilliantly stupid:

1. Run svn info on the main URL to see whether or not we are up-to-date.
2. Run svn log --stop-on-copy to get a list of all the revisions we need to download.
3. For each revision we don't already have, run svn co --ignore-externals.

Once we have a local checkout of the code, we store the revision into a local Git repository. Essentially, we have a very brute-force Subversion-to-Git converter. All of our code analysis will be run on our local Git version of the repository.

So if you can survive a call to info and log, and a checkout of each revision, we can swallow your repository.

This brute-force mentality was what we settled on over other more elegant, off-the-shelf methods precisely because its simplicity means that it works with the greatest fraction of existing repositories.

We do have another Subversion importer that uses svnsync to create a local mirror. It's much more efficient, but currently we only use it with a few select forges that we know support svnsync.

Making a long story short, I'll reschedule the download and we'll see how far we get. I'm optimistic. :-)

Thanks,
Robin

Robin Luckey about 16 years ago

Your optimism was correct; it's finished: http://www.ohloh.net/projects/3330?p=Achievo+ATK
Great! Thanks a lot for your support!

Ivo Jansch about 16 years ago

Heh, whew! After all of the anticipation, I hope there aren't any problems with the report itself. :-)

Robin Luckey about 16 years ago

Was anxious to see what it would report but so far it looks good. :-)

Ivo Jansch about 16 years ago