Forums : Technical Issue Help

Hi!

The project LimeSurvey/PHPSurveyor exists for 4 years now (and so does its code history).
The enlistement at http://www.ohloh.net/projects/270/enlistments says its only a few day old.
Can you please fix it?

Thank you

Regards

Carsten

Carsten Schmitz almost 17 years ago

Hi Carsten,

You've just run into a tough limitation in our SVN history parser.

On May 16, you moved your entire source tree out of /source/phpsurveyor into /source/limesurvey.

Our downloader only tracks the activity in the precise enlistment directory you specify -- in this case, /source/limesurvey. Since this directory did not exist before May 16, we show no history before this date.

Yes, this is pretty lame, and I don't think there's a workaround for you at this point. There are many projects with this problem, and this is a priority fix for us.

Robin

Robin Luckey almost 17 years ago

Hi there!

Just to state the importance of this bugfix: Same for project 6065.
I would really love to see this capability added. Maybe a simpler fix could be to let users add both paths.

Thank you,

Hagen.

Hagen Möbius almost 17 years ago

Hi!

A question to the development team:
Is this 'priority' fix still in the making?
Is there any ETA on this?

Carsten Schmitz over 15 years ago

Wow, time flies. No, I must admit that we still do not have an ETA for this, and we do not currently have it on our work schedule.

It's still a priority in the sense that it's one of the top two or three things that our users ask about. However, as a small start-up we have to focus our energy on things that might generate some revenue, so Subversion improvements seem to always be second in line.

Over the long term, we will probably refactor our source control adapter code so that we can open the source and take community contributions. We've had great results from opening up our line counter, but changes to the source control adapter would impact the core design of Ohloh's server farm and data schema, so it will be a while before we can address this.

Robin Luckey over 15 years ago

A simple, low impact fix (hopefully, but I don't know your code or server farm structure) that would probably make most people happy would be a way to add an enlistment at a revision. That way, you could create an enlistment of limesurvey@TRUNK and phpsurveyor@1234. The history parser should probably be able to backtrack from 1234 back in time to revision 0.

Peter Bex over 15 years ago

That sounds reasonable - Robin, pretty please?
It just looks so bad if 2-3 years of history is missing - for many other projects that would be the same.
Good thing is also that enlistments with valid revision numbers would have to be only crawled once.

Carsten Schmitz over 15 years ago

Yes, the idea to import a specific range of commits has been bouncing around, and it might happen as a stop-gap measure before we can do a complete fix. Even so, I can't make any promises about when we could get this done.

Robin Luckey over 15 years ago

Just to throw a me too on the pile. The OpenEJB history goes back to 2001 but shows up as started mid 2007 -- likely to be the case with any apache project that has gone through the incubator and the inevitable directory change that happens on graduation.

Would be nice to get even a rough hack in.

David Blevins over 15 years ago

And another me-too; HTML-Table-FromDatabase claims to have only one commit - this is the commit where I re-organised my repo to group things I wanted to expose under a common tree.

As it was moved with history, the previous commits are visible via svn log, but Ohloh doesn't see them.

(Admittedly this is a tiny project, so it doesn't matter much :) )

David Precious over 15 years ago

Well, also doing my me-too.

My prototype project with this issue is DerMixD.
At some time stuff has been moved from dermixd/trunk/ to trunk/ (that's what's left from migration out of some bigger CVS repo, even).

I must admit that I wonder what the ohloh code does to get the history... a simple

svn log -v --xml svn://orgis.org/dermixd/trunk

gives you the proper history with movements as far as subversion can tell.
Well, I am standing in line, waiting for the fix to get my commit count up;-)

Thomas Orgis over 15 years ago

Actually, mpg123 is a much nicer example for difficulties with code history.
You have movements plus a frankenstein-like history resulting from generating some initial svn history via diffs from release versions, plus integrating some CVS history (older commit dates, but newer code) and then actually committing stuff in svn.

What mpg123 also nicely shows is the problem of correct attribution: I created the initial commits from code written many years before by Michael Hipp... well, I could manipulate revision history to change commiter name and commit date (setting to release/tarball modification time of version x), but that would still leave open the problem of commits that actually have two contributors behind them (no fantasy, they are there;-).

Well, what I want to say is: Once ohloh gets the meaning of mpg123 svn history right, it is a good way near perfection;-)

Thomas Orgis over 15 years ago

Hi Thomas,

You're right that just reading the full commit history is pretty easy... after all, it's just right there in the svn log output waiting to be parsed.

The challenge for us comes in getting the actual contents of those commits. Ohloh was just simply not built to expect that the root directory (and therefore the Subversion URL) changes over time, and our system of retrieving the file diffs and counting the code relies on a stable URL.

It's not an insurmountable problem, but it significantly complicates the download logic, and it's going to take some dedicated effort on our end to get the required changes made.

Robin Luckey over 15 years ago

Yet another me too - same problem for Magnolia.
I wonder if providing links to old urls on our end would help as a workaround until you can fix the problem on your end?

Jan Haderka over 15 years ago

There was some talk about a new native subversion adapter that ought to be capable of doing this. What's the status on that?

Losing years of commit history info isn't fun :(

Peter Bex almost 15 years ago

Hi Peter,

We do have an improved Subversion importer now that can follow certain types of directory renames. Let me know what project(s) you're interested in, and I'll take a look.

Thanks,
Robin

Robin Luckey almost 15 years ago

Well, I'm a pretty active contributor of extensions (eggs) for the Chicken project. So far, I've created ohloh projects for the following (not all are affected by this problem, see below):

• chicken-postgresql
• intarweb
• uri-common
• uri-generic
• imlib2-chicken
• epeg-chicken
• spiffy
• phoghorn
• wmiirc-chicken
• web-unity
• chicken-9p

Now, Chicken's subversion repository has a pretty unusual layout where directories are copied for every major release, so that eggs for the previous release can still be maintained (each release's egg directory has a trunk, tags and branches). But there was also a large reorganisation a few years back when every egg (chicken extension library) was kept in the root of the repository.

For illustration, we can look at one particularly bad example; the chicken-postgresql project. It was first moved from /postgresql to /release/2/postgresql, which was then copied to /release/3/postgresql and then all files in there were moved again into /release/3/postgresql/trunk and then this was copied to /release/4/postgresql/trunk I'm sure you'll agree this is a pretty horrible thing to enlist.

I cannot manually get at two parts of this tree: The original /postgresql, and the /release/3/postgresql files before they were moved to /release/3/postgresql/trunk I had to enlist all three release entries to get Ohloh to properly see all 5 contributors, but now ohloh thinks the egg has three times the LOC count it should have (you can see huge jumps in the code graph). I'm not sure if it now sees the entire revision history, as there's no easy way to check this with Ohloh.

You can also view its history from Trac here (using the 'postgresql.scm' file which is a constant throughout its entire history)

Peter Bex almost 15 years ago

Oh, by the way: the wmiirc-chicken, uri-generic and chicken-9p eggs exist only since release/3 and have been copied once to release/4. The intarweb and uri-common eggs only exist in release/4.

I believe all the others I listed have been around since either release 2 or have seen the copy from /projectname to release/2/projectname.

For other people's eggs (yet to be added to ohloh), I'm not 100% sure they've all been nicely copied using svn copy, so it would be nice if there was a way to access the old directories using an explicit revision enlistment.

Peter Bex almost 15 years ago

The OP's problem doesn't seem to be fixed either; clicking on Limesurvey's established codebase factoid reveals that ohloh thinks its first commit is from 2006, not 2003 as would be correct (according to the post, the project was 4 years old at the time, and the post is 2 years old).

Peter Bex almost 15 years ago

Hi Robin,

thank you for finally picking up this issue. I see that my SVN-Sync for the LimeSurvey project is 'Waiting in queue' - does that mean you scheduled it now for the new analysis plugin?

Carsten Schmitz almost 15 years ago

Any news on this?
I am still wondering what happened to this 3-year old 'priority fix' for this issue.

Carsten Schmitz over 13 years ago

Carsten:

I've just created an issue for this in our bug tracking system.

I don't have any estimates as to when it may be addressed.

We're very busy here getting up to speed on ohloh's systems and preparing for the server migration, along with dealing with other issues.

I just wanted to reply and let you know that we see the request and will evaluate it once we are able to.

RobertSchultz over 13 years ago