Posts

Showing page 1 of 2

Ohloh General Discussion : Kudos rank calculation flawed?

Hi Robin, thanks once more for the clarifications and explanations, esp. the link to PageRank, and also about the zero sum principle. That makes a whole lot sense. As for projects not receiving ... [More] Kudos: Aye, you are right of course (no surprise, you are undoubtedly the expert ;) they don't. And I very badly expressed myself there. My point was more like this: If I stack a project, I assign a certain amount of kudos to each developer of the project. Another way to view this is that I assign some kudos to the project, which is then split between the developers of the project. So, while technically speaking I assign Kudos to the project members, not the project itself, from my point of view I didn't do any such thing, I gave Kudos to the project (I stacked it, thus marking it as "useful to me"). If I now look at the profile of a developer of that project, I do not see the indirect Kudos; I only see the direct kudos, and the size of the stacks. Hence if there are two people without any direct kudos, but each is member of a (different) project which has been stacked 100 times -- then I'd expect them to have equal Kudos rank. And am irritated if they diverge by several hundred ranks. So, as a conclusion, I guess what I really want to say is this: Naively spoken I would prefer if stacking a project would assign a fixed amount of kudos to the project resp. it's team members, treating all "stackers" equal. However, in light of your words on zero sum games, this might not be possible. Still, it would solve two problems: The "can't understand indirect kudos when looking at person overview page", and the "discourage people from making huge stacks" problem. As for you developer A vs B question: Tricky one, really. In the end it boils down to defining arbitrary weights, resp. a philosophical one: "Which color is bigger? green or yellow?" well you just can't compare colors that way. Neither can you really compare "stacking" vs. "Kudos", as the relation is different for everybody. It will always require a somewhat arbitrary decision to weld together the two concepts and make them comparable by force ... If being stacked by 100 people is well above average, then I'd feel this should give a quite big kudos boost. But not as much as direct kudos from 10 "important" devs -- direct kudos is the true kudos, it means "yes, this guy did great work". While stacking a project means "Hey, this project is great", and only by indirection implies the devs (or at least some of them) can't be totally wrong either, given that they developed this great app. I think this is also one of the core difficulties with KudosRank: It mixes two different concepts and tries to fit them under a single umbrella. More, if you also try to factor in the "activity level" of a project and its member by considering the commits counts... Cheers, Max P.S.: Any chance you guys could allow for a little bit wider columns in this forum? It just doesn't seem to be made for people like Robin and me who prefer writing "too long" posts ;-) [Less]

Views 6141

Author fingolfin

Posted almost 8 years ago

Technical Issue Help : Download probs & suggestion

Hi there, according to http://www.ohloh.net/projects/scummvm/enlistments, the ScummVM repositories were last scanned 28 days ago. In addition, it gives for ... [More] https://scummvm.svn.sourceforge.net/svnroot/scummvm/scummvm/trunk the following message: "Step 1 of 3: Downloading source code history (Failed)". Is there anything we can do about that? In particular: Can I somewhere find out what the error was, so that I can decide in the future (read: for the next time this fails) whether I am able to fix it, or whether I have to write to this forum ? If the Ohloh staff can unwedge this, I'd be grateful. Now the suggestion: The page also says "Please wait about 24 hours before reporting a problem". However, how should I know whether a failure happened 24hours or 24 days ago? So, how about adding a date to these messages: I.e. don't just tell us that a step failed, also tell us when ;-). Thanks! [Less]

Views 443

Author fingolfin

Posted about 7 years ago

Ohloh General Discussion : Support for mercurial

+42 (I figure it'll be faster this way? g)

Views 17297

Author fingolfin

Posted almost 8 years ago

Ohloh General Discussion : Kudos rank calculation flawed?

Hi there, today I was randomly looking through the Kudos rankings. I figured that looking at people with high Kudos might reveal some interesting new projects etc. Doing so I observed something that ... [More] to me (after reading the "How is Kudos rank calculated" description) makes no sense. Essentially, I would have expected the following to hold: "If a person gets Kudos from others, and participates a lot in many high-profile projects, (s)he will have a higher Kudos rank than somebody who got no Kudos from others, and contributes less to fewer, lower-ranked projects". However, that doesn't seem to be the case. I could use my own profile for comparison now (which, to be honest, was my initial reason to write this post -- yes, I am a Karma whore :-), but that's not necessary. To give an example, consider these two persons: 4267 (rank 494) 4197 (rank 588) 183 (rank 595) I don't understand how Mr 4267 can be 100 ranks higher than Mr. 183). That seems to make no sense -- but maybe I am just missing something? The most extreme example, however, is this guy: 4513 (rank 400) who has done nothing (well nothing visible on Ohloh at the time of this writing), except for getting Kudos from somebody on Kudos level "1". What?!? Would be nice if somebody could try to explain these particular examples, and maybe the Kudos ranking description could be clarified a bit. :-) Ed: linkified post. [Less]

Views 6141

Author fingolfin

Posted almost 8 years ago

Ohloh General Discussion : Kudos rank calculation flawed?

Hi Jason, thanks for the reply, I now look forward to maybe hearing from Robin. I know how difficult it is to get a ranking formula "balanced" (SourceForge.net took years for it, and the original ... [More] formula was totally flawed). Esp. when you use an iterative definition as you do, or as Google's pagerank uses, it's a hell of a time to get those tuned, and in the end you'll always get some data which is "surprising". There's a reason this kind of algorithms comprises an interesting research field of its own in comp.sci. and math ;-). So good luck on this one. (Oh and thanks for linkifying my post.) [Less]

Views 6141

Author fingolfin

Posted almost 8 years ago

Ohloh General Discussion : Kudos rank calculation flawed?

Hi Robin, thanks for the info, that answers most of my questions. Being a mathematician I am always interested in mathematical details ;-). I dealt a bit with the theory of "fair" voting systems in ... [More] the past, which has quite some relations to the problem at hand. In particular I was reminded a bit of Arrow's theorem. For the 1/n rule, let me point out another possibility (I am not trying to promote it, just pointing out another possibility): You could use a "non-linear" scaling, e.g. make the influence of a Kudos proportional to 1/log(n+1) (base 2 logarithm used here). This way giving a single kudos counts fully, giving 16 counts 1/4th, etc. And then of course you could combine this with a capping scheme, too... And other formulas are possible as well, of course. In fact, 1/sqrt(n) might be a good one, too. This would in fact be in line with some results in election theory which imply that the total influence of a group among several groups (think of the population of one state out of a federation of states, like in the US or in the European Union) grows proportional to the square root of its size, instead of growing linearly as one would naively assume. But I didn't do anything to properly research the connection to this result from election theory, it's more a gut feeling, so don't take it too seriously (but it might be worth trying this out anyway). The alternative of not doing any scaling of the kudos a person gives (i.e. each counts full) is appealing because of its simplicity and clarity. Your voice counts fully, no matter what you do. However, this is not without problems, too, as televoting or webpolls show (or, for that matter, democracies): A few dedicated persons willing to make sure there "voice" is heard can spoil the system. In our case by spending kudos on lots of persons. Even "kudos bombing" would be possible: Create lots of fake accounts, give kudos to each other, and then use this accumulated Kudos power to boost other people (why on earth you would want to do that is beyond me, but my lack of imagination won't stop anybody ;-). However, this is just randomly playing with the parameters, without real understanding of what's going on. It might be worth to do a bit research and look at the literature whether there are some applicable research results and/or discussions of similar problems... As for the project kudos: This also seems "bad", but is likewise probably not easy to fix (and possibly even impossible to fully "fix", see ). It "seems" queer that a single contributor to a low profile project would get more kudos than a high profile contributor on a project with many other contributors. But I do understand now what's going on there. This is indeed a difficult problem. Wonderful, I love difficult problems, esp. those which aren't mine ;-). Cheers, Max [Less]

Views 6141

Author fingolfin

Posted almost 8 years ago

Ohloh General Discussion : Kudos rank calculation flawed?

Another potential problem I just noticed today (as I experienced it myself): Get a couple of your friends to sign up, say three. Then, each of you gives kudos to each other. Don't take too many. Now ... [More] each of you got kudos from three people, so you get a nice boost in the current system (due to the current "sparse data" situation). Hence you get a nice kudos boost. Which increases the worth of your kudos, which means that your buddies get a bigger boost, which means you get an even bigger boost in the end. I think it's important to balance the system against this kind of use. This is, in this situation the diminishing effect giving kudos to three people should be balanced against the boost these people get, and the back-boost you get from them in turn. How much does the kudos rank factor into the kudos I give, anyway? Is the kudos of a rank 9 person 9 times higher than that of a rank 1 person? Hm. [Less]

Views 6141

Author fingolfin

Posted almost 8 years ago

Ohloh General Discussion : Kudos rank calculation flawed?

Actually, I don't think it's impossible to get around "loop abuse", i.e. people pimping up each other by forming a complete graph. The problem here is that so far the Kudo ranking has been developed ... [More] "from the guts", and not by applying rigorous mathematical arguments. This is prone to break / not work, as sometimes "common sense" just fails to find the "right" solution, or even comes up with solution that seem "correct" but actually can be shown to be worse than most other algorithms. Which is why I proposed that one should at least try to look at the literature, I'd not be surprised if there weren't already some papers etc. dealing with similar problems. As for the stacked-project-kudos, it makes for an example were one could challenge the whole approach. Right now, you have taken an "user centric" view point -- you check how many projects got kudos from a person, and then assign to each some value. But IMO a "project centric" view is also possible, and actually is the one I assumed in use (until I learned in this thread that I was wrong), and which lead me to my "wrong" assumptions in the first post on this thread. Namely, if I look at a project page, and see that 500 people stacked it, I think "Wow, lots of people use it" and would expect that it gets a big boost from this, compared to a project with say 10 people who stacked it. And for this I do not care at all what the Kudos rank of the people who stacked it is. So if 100 anybodies use project X, while project Y is only used by the the top 4 (according to Kudos rank) developers -- then I definitely would rank X higher. Probably with 10 vs. 4, too. I guess below that I could accept a certain higher influence of the "high profile" devs, but I wouldn't complain if all users were treated equally on this. At most, I'd say a Rank 10 dev should get maybe 20-50% more influence, I'd say (from my guts, too, I admit :-). Going further, assume project A and B are stacked by 10 people, but the 10 stackers of A each stacke 100 projects, while those 10 stackers of B only stacked B and nothing else. Then I would expect that A and B get the exact same boost. I simply see no justification to do it differently... :-) Just my two cents, and again, I didn't sit down to work out on a rigorous theory behind this, so I don't claim to be "right" or anything, just stating my thoughts here. [Less]

Views 6141

Author fingolfin

Posted almost 8 years ago

Ohloh General Discussion : Kudos rank calculation flawed?

OK, I just noticed a new wonderful example which highlights some problem with the KudosRank: Take a look at Mr. 3409. He is ranked 565th at the time I am writing this. He neither received nor gave ... [More] any direct Kudos. However, he performed 98% of the work on the single project he is member of. This project was stacked by four people -- one Mr. 3409 himself (he's a 9er), and three level 1 guys. Compare this to e.g. my buddy Mr. 803, right now only at 1344, he's on level 8. Yet he got direct Kudos from three people, and some stack Kudos. Just weird. I assume the system handles the case where Mr X is in project Y and stacks it, so that he doesn't give himself Kudos that way? Does the system [Less]

Views 6141

Author fingolfin

Posted almost 8 years ago