Howdy, Stranger!
Categories
- 4.8K All Categories
- 4.3K Life in Lusternia
- 453 Announce Posts
- 75 Event Posts
- 1.7K Common Grounds
- 589 Q&A
- 83 Combat Overhaul
- 1.5K World Library
- 86 Combat Logs
- 870 Event Scrolls
- 403 Mechanic's Corner
- 329 Ideas
- 314 Last Chance Trading Post
- 478 Life Outside of Lusternia
- 9 Forum News
- 275 The Real World
- 94 Meet and Greet
- 37 The Funnies
- 63 Mafia Hideout
Reporting System 2.0
I think at this time, it's safe to say that the new reporting system that we implemented hasn't really panned out and doesn't really feel effective.
The general feeling is that it is arduous to use and takes up quite a bit of time to be implemented and it can be incredibly frustrating when you spend time drumming up support and effort getting people to vote, just to find it doesn't reach the threshold or it gets rejected. There's also the fact that because there is a threshold involved that needs to be met, players are not interested in voting for what they really think, but instead voting to drive the report to the next step, even if they don't agree with it.
With those general feelings in mind, I've come up with a new idea to adapt the current system into. I want to note that this would be an easy adaptation, and would not interfere with finishing off timequakes or starting on the mage revamp. It's something that can be done while waiting for feedback on mage proposals for instance.
Reporting System 2.0
Reporting System 2.0 would remove reporting being open year-round and instead would be integrated into the development process. We would strive to open reports up 2-3 times a year (every 4-6 months) and once they are submitted, we would take time to make the decisions on the reports (which would be dependent on the number of reports submitted) and once decisions are rendered, we would then immediately work on implementing those reports that are approved.
The submission process would be two stages. The first stage would last about 15 days. During this time, anyone who meets the requirements (currently 5 vote weight and > lvl 30, but up to debate) can place up to 3 reports. This again, would be by registration and not per character. During the first 15 days, reports can be edited and changed and what not, but would not be able to be commented on or voted on. They would still remain anonymous.
After the first stage, any report submitted automatically goes into the second stage. The second stage will last another 15 days. Reports will no longer be able to be edited. At this time, players can then vote on reports. Voting options would be Support, Support with Changes or Reject. In order to vote for Support with Changes or Reject, you would need to submit a comment first explaining why. Every player will be able to comment once on a report. All votes and all comments will remain hidden from players. This is in order to stop the envoy war style fights that erupt on reports.
Once the report finishes the second stage, it will be decided on. The three decisions are accept, reject and hold. Accept means we accept it and will implement it. Reject means we will not implement it and Hold means that we like it, but we are not ready to commit the time it takes to implement. We will not give a timeframe on when a hold report will be implemented, it will just be indefinite status. It could be that we do a major rework and are able to work it in with that, or we decide we want to implement it. This category is in response to us rejecting reports that we like but don't have time to do. A kind of middle ground.
Once decisions are rendered and accepted reports are implemented, reports will then be closed until the next time we open them.
If we do decide to go this way, then we'll like just turn off the ability to report things currently and let whatever is up through the current process. I would strive to implement every report we've approved through the current system before moving to a new system.
Let me know your thoughts.
The general feeling is that it is arduous to use and takes up quite a bit of time to be implemented and it can be incredibly frustrating when you spend time drumming up support and effort getting people to vote, just to find it doesn't reach the threshold or it gets rejected. There's also the fact that because there is a threshold involved that needs to be met, players are not interested in voting for what they really think, but instead voting to drive the report to the next step, even if they don't agree with it.
With those general feelings in mind, I've come up with a new idea to adapt the current system into. I want to note that this would be an easy adaptation, and would not interfere with finishing off timequakes or starting on the mage revamp. It's something that can be done while waiting for feedback on mage proposals for instance.
Reporting System 2.0
Reporting System 2.0 would remove reporting being open year-round and instead would be integrated into the development process. We would strive to open reports up 2-3 times a year (every 4-6 months) and once they are submitted, we would take time to make the decisions on the reports (which would be dependent on the number of reports submitted) and once decisions are rendered, we would then immediately work on implementing those reports that are approved.
The submission process would be two stages. The first stage would last about 15 days. During this time, anyone who meets the requirements (currently 5 vote weight and > lvl 30, but up to debate) can place up to 3 reports. This again, would be by registration and not per character. During the first 15 days, reports can be edited and changed and what not, but would not be able to be commented on or voted on. They would still remain anonymous.
After the first stage, any report submitted automatically goes into the second stage. The second stage will last another 15 days. Reports will no longer be able to be edited. At this time, players can then vote on reports. Voting options would be Support, Support with Changes or Reject. In order to vote for Support with Changes or Reject, you would need to submit a comment first explaining why. Every player will be able to comment once on a report. All votes and all comments will remain hidden from players. This is in order to stop the envoy war style fights that erupt on reports.
Once the report finishes the second stage, it will be decided on. The three decisions are accept, reject and hold. Accept means we accept it and will implement it. Reject means we will not implement it and Hold means that we like it, but we are not ready to commit the time it takes to implement. We will not give a timeframe on when a hold report will be implemented, it will just be indefinite status. It could be that we do a major rework and are able to work it in with that, or we decide we want to implement it. This category is in response to us rejecting reports that we like but don't have time to do. A kind of middle ground.
Once decisions are rendered and accepted reports are implemented, reports will then be closed until the next time we open them.
If we do decide to go this way, then we'll like just turn off the ability to report things currently and let whatever is up through the current process. I would strive to implement every report we've approved through the current system before moving to a new system.
Let me know your thoughts.
11
(c) Iron Realms Entertainment 2012.
Comments
theatre due to the snowy weather.
hungering malice.
As for contentious reports, this will do away with that and I firmly believe that 'counter reports' should be tossed out. There's trying to fix something to bring it inline, then there's just being malicious because someone is touching your toys. Case in point, the Night report 04191101 has so much infighting and threats of this, when most of the comments aren't even addressing the solutions, and most just outright not being helpful to the process. We need to be able to place reports on one another's skills, because trusting players to care about how imbalanced their skills are seems to be too difficult.
Thus, I'm fine with opposing skillsets being reported on at the same time, and we'll definitely know which reports are just retaliatory because of a 'contentious' report, in which case it can either be reviewed and allowed, or be taken down like the last 'counter report' was.
theatre due to the snowy weather.
hungering malice.
The limitation to a few times a year is kind of the compromise on removing the thresholds. One of the major goals of using thresholds was to limit what reports got through so there was less work on our side of things. Ideally, the reports that the community at large wanted, were the ones that made it through. While this led to situations where people felt they couldn't get their reports through (and often would just keep submitting it until it did get through), it was an effort to reduce workload on our end and hopefully prevent situations where reports sat in limbo for up to a year (which is what was happening under Envoys). So by removing thresholds, the compromise is that we're going to limit to only a few times a year but allow for more reporting and guaranteeing that each report gets looked at.
As far as scope and scale. Again, I don't want to limit what you can and cannot report (outside new design templates and other things that have cropped up). I really want reporting things to be something that everyone feels like they can suggest and contribute things to make Lusternia better. I do think moving to a system like this would enable players to put together a comprehensive set of reports that could work to improve a class overall. To be upfront - how easy something is to implement is the second most important facet taken into account. A lot of times it's weighing the overall benefit to the effort required to implement. So a series of reports that makes small changes to a skillset to improve its viability is much more likely to be accepted all together than a series of reports that require new mechanics and complete reworks of skills. Again, this isn't to say you can't report it and that it won't be approved, but just to make things transparent about our process.
In regards to comments, based on the posts here, I am considering allowing comments during the first stage, but again limiting it to one comment. This is tricky because I don't want arguments and fights erupting on reports. I also don't want people to edit their one comment 15 times to keep responding back and forth that way. I am aware that by making comments hidden, we're more likely to get these retaliatory 'tit for tat' reports. Honestly, there really isn't any way we're going to reasonably prevent malicious reports because it's going to be difficult to prove intent.
One thing I've noticed lately is that there's been a lot more of looking at mechanics and skills in a vacuum and saying things like 'well class A has it, so it should be fine that Class B has it too' or 'If you're taking it away from Class C, then take it from Class D too.' Historically, this kind of logic and reasoning has always been shot down. Just because Class A has it is not a reason for or against class B having it either. Removing it from Class C is not a reason for or against removing it from Class D. We shouldn't be looking at specific mechanics in a vacuum but instead taking the whole skillset into consideration when proposing changes. You should be able to justify your argument with more than just 'such and such class can do this so I should be able to as well.'
With that in mind, we'll probably just limit the required response to support w/ changes and allow both accept and reject without commenting.
I don't like the inability to comment or vote in the time before submission or the 15 day workshop period, which is useless without those two things anyways. If all of the thresholds are going away I don't understand the need to limit these things, just give report authors more tools to moderate their reports and measure the support.
1) some formal means for players to raise big issues that might crop up outside of a reporting period. This could be as simple as having that be a function for the envoy channel, or for some kind of off-period voting mechanic with a high threshold (adjusted for factionalism), or even just someone scanning over the extant reports being workshopped to pick out scary problem statements.
2) Some function of envoy input/filtering to help further prioritize reports. That's basically what the envoy systems have always been, a way of filtering and prioritizing change requests. Either by having a bunch of ostensibly experienced people take in the comments and suggestions from their orgs and adjudicate which go to report, or through the democratized voting threshold process.
Perhaps at all of the stages, have envoy comments marked out for the purposes of providing factual/experiential input. Or at the end once the report is finalized a envoy polling vote opened for a bit to push to the front the more impactful reports. My understanding is that this is a feature of the Classlead system in the other games, that don't require "votes" as much as a seconding (and possibly thirding) in the manner of designs being mortal reviewed. Not sure exactly what form this input would take, but I think it's a waste of the resources volunteered by envoys to do otherwise.
I suggest the following setup instead of the extant one posted in the OP of the thread. Most of the numbers are just random inserts - except for the 3 reports per active person submission limit.
Reports Phase One
----------------
This phase will be constant. No window.
Each player above a certain threshold can create and maintain up to 5 total reports. Any 3(?) of these can be "Under Consideration". This works similar to how the consideration phase does now: it goes up for other players to see, comment on, vote. These comments and votes are purely for the benefit of the report writer's process. At any point players can take their reports back from consideration to draft to edit them, change around solutions, wipe comments and so on.
Personally, I don't like anonymous comments at this stage, they should at minimum be given a consistent label within the report (aka, the first commenter is always called "Bob" and then at any point that they comment or vote on that report it's labeled Bob and you know it's that one person again). [Edit: The report owner would be marked as such, too]. The whole point is to give tools to decide how good the solutions are and how much honest support they're likely to gain, being able to go "Oh, it's not actually 10 different people saying they don't like this solution in the comments, it's one really loud guy who is giving everyone so much of a headache they don't want to even think about this report" and... ignore them. Heck, I think it'd be a good idea to give some measure of moderation control to the report writer to turn comments on or off, or even specific commenters on and off - at this point it's entirely for their benefit to take or not take comments to improve the report.
I would put a cooldown on shifting reports to or from Consideration, so you aren't just constantly bouncing them up and down all the time to give everyone a headache and keep refreshing comments. Also, players should be encouraged to delete reports they decide against and start a fresh report for a new idea instead of repurposing the same number - so that comments can be archived for admins just in case and not be a total mess.
EDIT: Perhaps do that nifty comment nesting trick, but instead when the report is taken down and edited have there be a line at the top of the comments with reversion logs that when clicked/input expand out to the older versions of the report and its comments?
Reports Phase Two
----------------
15 day window to decide what to set. 15 day window to vote and comment.
In this phase, players select up to 3 of their reports to set to Pending. Comments are somewhat limited, and invisible. This is everyone's last chance to summarize their concerns about/support for the report. In the locked period envoys are expected to weigh in if there's any weird niche interactions related to their org and its classes with comments/input. Perhaps some second layer of semi-visible comments called "endorsements"?
It might also be good to look into how classleads work in the other games. If they have a small upvote threshold it might be possible to open up the number of reports per person but have some other metric by which they're funneled down from there.
EDIT: also, I'd bring back specifically selected envoys from each org for this purpose, not just the elected leaders. Perhaps default to that but allow them to instead nominate someone else from their org to be the envoy.
I'm not really sure that allowing players that post reports the ability to turn on and off comments or really moderate them in any way is a good idea.
I'll think about having a general open period where players can put up reports for discussion.
The biggest problem is most reports turn into:
* Submit report that may or may not be needed.
* Make case.
* Someone counters with unrelated point. Some discussion begins to follow this.
* Spam comments result.
I would like to suggest a 2 month cycle. Odd months would be a period to submit reports. During this month (which should be made by the 15th or required finalizing before the month starts as a topic), people merely discuss the idea, not the solutions. This is to come to a reasonable understanding that there is a problem and then work out how to tackle it. Conversely, we state a problem and our chosen solutions, which may or may not cater to the problem given. At the end of this period, Oversight (Orael and co) agree or disagree that there is indeed a potential issue.
In even months, you create the actual solutions and implement discussion on the specifics of the answers.
I acknowledge that this may be more complicated, but the biggest issue is when points degrade or turn into back and forth. Establishing there is a problem is not the same as having a solution, and I would love for there to be a requirement to find a potential issue first. I wouldn't mind a "furies thoughts" on the decision to move to stage 2 or not. We currently have debate on both the problem itself and the solution.
/unpopularIdea
I'm also not sure that a "true collaborative" approach allowing direct addition of solutions to reports is a good idea at all. Sometimes a problem statement has a variety of 'solution's that slightly alleviate the problem but don't address the full breadth of the issue, or indeed that appear to solve the problem but cause other issues/actually make things worse in practice. The addition of these solutions can cause problems. People would be free to make those suggestions in their final (closed) commenting period if so desired.
The idea with having a wide time in which to put up a report, take in comments, revise the report, and iterate until you're happy with the result. "No" votes won't mean losing time on the window, or failing to meet a threshold by dragging down momentum, just some departure from the suggested solutions. Personally, it's why I'd like to have at minimum some kind of persistent/masked labeling instead of just totally disconnected and anonymous comments/votes, if not just attaching everyone's names to some parts of the system. Not report writing and submission, but input? If the report writer notices "huh this player just votes no in this pattern regardless of the contents" they should be able to take that fact into account when considering adjustments to their report. As noted, a minimum of internally consistent masks applied to comments and carried over to votes would be great. Again "huh, someone voted no on all solutions but they had no comment/their comment had nothing to do with the content of the report" type situations would then be visible.
I also know that given 3 reports for months on end and wanting them to be good ones I feel like I'd really want some feedback and more than a week or two to get it, not knowing if the next cycle will be in 3 months or 6. The only way to get that without some kind of commenting mechanic would be to go to the forums [or the much vilified discord] at which point the anonymity is silly anyways.
As a sidebar I do think it's a bit of a problem to be having people make arguments against your report (presumably) and not have those be in any way visible to you and just... trust administrators to sift through what could be common misunderstandings that they just happen to share, or misdirection they fall for by virtue of being busy. Neither of these things are without precedent in the course of reporting history.
Plus, I'd totally down for them coming up with unique sol 4s that give new counterplay instead of some hotfix us players come up with.