Map Ratings

by Pudd • December 08, 2014
Please consider including the feature of having difficulty ratings for maps, or single jumps if a map is made up of independent sections. Something similar to the way most rock climbing scales work would be fitting I think, where there is a minimum level on the low end of the scale, but the high end of the scale is open-ended. For example, with the Hueco scale in rock climbing, climbs can only be rated as low as V0, but can be rated as high as necessary (V16 is the hardest climb currently, but maybe someone will become good enough to do a climb deemed hard enough to be a V17).
There are a couple advantages for a rating feature. One is that players have more indication of progression. Maps/jumps they completed could be saved checklist-style in the game or on an external site to further encourage progression. This is especially true for newer or less-skilled players who don't have much hope in competing for times. The other advantage I can think of is that it serves as a guide to players when looking for new maps to try. Usually players will want to do jumps that are at their skill level, or slightly above it. Or maybe they'd be in the mood to do something a little below their skill level to relax more.
These ratings could possibly be further broken down to the core components of what makes jumps difficult. These components should ideally be independent of each other. Here's a possible list (please suggest more if you can think of anything else):
    • Strafing. The lowest score means you can just jump normally to complete the jump. A high score means you need a lot of strafing skill to complete the jump.
    • Precision. A low score means that aiming your jumps correctly is not a huge factor in the jump. A high score means there are very particular locations you need to land in order to complete the jump (think paper-thin platforms).
    • Endurance. A low score means that the jump is very short. A high score means that the jump continues for a long time without moments to pause.
    • Complexity. A low score means that there is at least one obvious and simple route to complete a jump. A very high score would mean that there is only one specific way to complete the jump, and it requires a high density of input changes and very specific timings for the input changes.
A map/jump's overall rating should be at least as high as the highest rating of a particular component.
Having these components of a rating would allow people to see what the strengths and weaknesses of a player are. This can incentivize players to work at their weaknesses, or allow them to focus on maps that have high ratings in what their strengths are in when setting times.
One issue is who would determine the ratings for maps/jumps. Some possibilities are:
1) The mappers themselves decide the ratings
2) Game admins determine the ratings
3) All players can pick a rating, but only once they've completed the map/jump. The map/jump's rating would be an average of the individual player ratings. It would probably be best to disallow new players to rate jumps, since they won't yet have a feel for the rating scale.
And one final thought - there can also be an enjoyment rating for maps/jumps. These I think definitely should be determined by all players and not just mappers or game admins.
Here's an example graphic of what would be generated for the "Curves" jump in the Mr. Men map in Enemy Territory:
Image
A little bit of info about who I am since this is my first post and most of you probably don't know me... I've been trick jumping in ET on and off since 2003 under the name Pudd. My strafing's pretty crap compared to others who are still jumping though D*=. About a year ago I discovered the timeruns server, which I was exhuberant about since I had been wishing there was a speedrunning community in ET. For the most part I do VET runs, but I also have done some AP.
ADMIN
Dinius • December 08, 2014
Wow, nice post. Short answer: yes. :mrgreen:
I have some notes here and there for such rating stuff, but currently, to keep it simple, I've only been working at it from this angle: 
Easy - Walkable, anyone can finish these maps
Medium - Mostly walkable, but some parts might require some basic strafing\ramping skills or alike.
Hard - Not walkable, many players might not be able to finish these maps.
But you've made quite a few good points. The reason I "abandoned" my original thinking with it was because I didn't see any really good way of achieving consistent ratings for maps. But you've offered up some good ideas for that too. I think it would make most sense to let mappers set some initial rating, which would then be added to \ adjusted by admins if needed. And like you say, allowing more "experienced" players, or at least disallowing relatively new ones from voting, would be a good solution to that.
I expect mostly everything you posted will be implemented, with some adjustments\additions here and there probably :) Again, nice stuff! :D
VIP
QaleQ • December 08, 2014
Maybe the game could calculate different things (see below) and come with a suggested rating for the map at completing it for the first time, but you're free to adjust the preset values aswell. I believe this would make it easier to get a more standardized idea of what difficulty deserves which rating.
Taking things like your track record into account whilst suggesting a rating. Thought I don't know if you'll be calculating skill somehow, but per-player skill could also be split up into those different cathegories brought up in OP. And if so, you could use the average number of tries and time spent on map (before first completion) compared to the different skills of the players that's played it.
Maybe the tutorial could be used to give you an initial skill-rating, presenting tricks from each  cathegory, with varying difficulty, and meassure how long/how many tries it takes for the player to complete them.
Might require a lot of work, and these ideas could probably use some fine-tuning..
ADMIN
Dinius • December 08, 2014
Tutorial skill rating is easy enough. But I cant see any way of making an auto detect type of thing work good. Even if I were to put a lot of work into it, it wouldnt be reliabe due to all the map design possibilities etc.
Pudd • December 08, 2014
I think that waiting for a large enough sample size to try or complete a map would take too long to get a reliable rating. If three high-skill players (let's say they have complete several maps with a rating of 15) completed a map on their first try, does that mean the map should get a rating of 0? 4? 8? Also, players can get lucky or unlucky with trying a jump. Or maybe it is easy for them, but they are trying it with the intention of setting a good initial first time, so they reset 50 times before actually completing it.
VIP
QaleQ • December 08, 2014

I think that waiting for a large enough sample size to try or complete a map would take too long to get a reliable rating
by Pudd

Depends on which rate maps are added to the game, but yes, this could very well be true.

does that mean the map should get a rating of 0? 4? 8?
by Pudd
Since it only affects the suggested rating, it would naturally be low if they completed the map their first time. We could only hope they have good enough insight into how skilled they are, and wish to have ratings as accurate as possible, so they can adjust the suggested rating accordingly.
Or maybe it is easy for them, but they are trying it with the intention of setting a good initial first time, so they reset 50 times before actually completing it.
by Pudd
Yea, this could be an issue indeed. What I'd like to see is that people get rewarded for completing maps in some way (even with bad times), so that they don't hold back on just trying to finish it once. Haven't fully thought out how that could be achieved. Maybe an option to discard the time could appear (at first completion only). But we'd need some way to motivate people to make their first goal to just finish a run, for sure..

ADMIN
Dinius • December 08, 2014
Think some more about that.. I previously thought it could be cool with some kind of point system or something, but haven't figured out quite how\etc.. Even just some kind of general "skill ranking" for players (ala League of legends), that isn't (solely?) based on good record times. This could be based on maps completed, your results in CTF \ TAG \ CIRCUIT games, and so forth.. :)
VIP
Piercy • December 11, 2014
Where would my map come into this? For instance, I've put a lot of time into my map making it possible for everyone to complete.  However, the fastest runs are pretty damn hard! If i get a seasoned Rik player looking for "hard maps" and completely missing out on mine.  At the same time i dont want a not so pro player avoiding my map because its too hard.
ADMIN
Dinius • December 11, 2014
Well many maps are like that, easy to complete but has harder routes, even if they're less "obvious" than in your map (i.e. a route can be hard without actually being that much "different").
 
It's obvious a rating system such as this can't be 100% perfect in every case. Has to be more of a "guideline" of sorts, with only the easy\med\hard yours would go in medium. There are some parts which require some skill, like the sloperun at start so it's not "easy". But there are no actual "hard" parts needed to reach the end. (compare to hard1 where you _need_ decent strafe and walljumps in order to reach the end).
Pudd • December 11, 2014
Yeah I thought this might be a problem, how some maps you can simply walk to the finish, but setting a good time on is hard. The rating system wouldn't be applicable on such maps.
Maybe it's best to create two different game types: trickjump and speedrun. Some maps would only be available in one of these game types, while others would be available in both. The rating system would only be implemented in the trickjump game type.
VIP
QaleQ • December 11, 2014
Yeah I thought this might be a problem, how some maps you can simply walk to the finish, but setting a good time on is hard. The rating system wouldn't be applicable on such maps.
Maybe it's best to create two different game types: trickjump and speedrun. Some maps would only be available in one of these game types, while others would be available in both. The rating system would only be implemented in the trickjump game type.
by Pudd

 Rather than those dividing up in just trickjump and speedrun, maybe some way to intertwine skill rating with the map rating system, using the same cathegories as used there and base skill on the maps rating, dividing skill into those 4 cathegories. Allthough, then the map ratings can't completely decided by just player-rating (people may rate a map unnecessarily hard just to make their first place weigh heavier), but rather use those as guidelines for admins (or something), that will have final say in which rating the map actually gets?

ADMIN
Dinius • December 11, 2014
Methinks we're straying into weird and too complicated rating ideas now :bear:
VIP
huPo • December 12, 2014
some maps you can simply walk to the finish, but setting a good time on is hard. The rating system wouldn’t be applicable on such maps.
by Pudd

 Yes it would, those maps would be rated as easy. If the map can be walked to end, it's easy map. It's up to you if you want to do it harder than needed. Map difficulty should never have anything to do with how hard the fastest route is.

Pudd • December 12, 2014
Yeah I guess defining a rating of 0 as being walkable makes sense.
Jorge • December 12, 2014

Here's my 2 cents:

Probably the easiest and most reliable way to rank a map is for the mapper himself to rank it, since he's the one making it, he'll be the one knowing first hand the difficulty of the map.

Or if that's not reliable enough there's also a possibility of having some players chosen to test the map and judge for themselves.

Pros: No need to get ridiculous amount of work in applying such a ranking system to the game itself.
Cons: Might not be 100% accurate since it'd be up to the ones ranking it.

ADMIN
Dinius • December 12, 2014
Actually that's not true @mapper decides imo..
1: Just because you map doesn't mean you're even a half decent jumper. You can make something you think is extremely hard while a more experienced jumper could do it easily.
2: Mappers could intentionally set inaccurate difficulties
 
 
 
Immediate plan/thinking is to just use easy-medium-hard kind of deal. Mappers can give their opinion, then me\hupo\arcaon\etc\any-other-people-we-learn-can-give-objective-and-relevant-ratings will set a final rating on it :)
The more detailed system isn't that important right away, but still, the stuff outlined in the first post is very valid. But it's better to have a "trusted" group of people decide on it. The ratings will be consistent and free of trolling and alike :p