KuroshinSubmissionQueue

MeatballWiki | RecentChanges | Random Page | Indices | Categories

The SubmissionQueue? at KuroShin [1] is how articles are submitted to K5 for review and voting on or off the site. It has a number of increasingly severe problems. This page is meant to serve as a discussion space for some of these, as I'm trying to formulate a Meta post to K5 itself regarding them. And yes, the intent is a fix. What's necessary first is a clear identification of what the problems are, how they're interrelated, and what needs to be done to solve the problem.

I've been spinning this for about the past nine months, really hard since the first of the year. Part of the problem is coming up with the stats and/or arguments to convince RustyFoster of what's going on and why it needs to change. There's a couple of things I'd like to be able to show convincingly. I also need to pull the latest CVS version to see how things are being done in code -- I'm relying on docs at the moment.

And yes, this preamble should probably get refactored down, but this is right now a cross between a personal page, a rough draft, and an anguished cry for help. So bear with, thanks.

This content is being added to incrementally, if you're interested in the order of things, check the diffs.

An awful lot of this is first person and subjective. Don't let this stop you from making comments.

If you've been drawn here by posts, links, or mentions elsewhere on the Net and are unfamiliar with Wiki's, take a look at TextFormattingRules and StartingPoints. Yes, you can edit this page, no you don't need an account (though it is considered polite to UseRealNames), yes, there is versioning (so we can undo changes if mistakes or mischief occur). No, there is nothing sacred about any of this text. Hop in.

"XXX" indicates a derailed thought train or more data to be added.

-- KarstenSelf 21 Apr 2001

[in fact, it is necessary to UseRealNames]

Some language issues

Submission refers to articles as they are in the submission queue.
Article refers to a submission that's been disposed of.
Disposed needs a replacement word, something like "adjudicated", to indicate "a final action has been taken regarding this matter". I've been looking for this word for about a year and it keeps slipping my mind. (perhaps "ruled upon"? I don't know if the judicial vibes would be approved upon, though... -- sayke)
Story shouldn't be used here, it indicates I've slipped and should replace the term with "article" or "submission".

Problems with the K5 submission queue:

: Suggested format: add problem to list here. Describe briefly -- a few words, or a sentence at most. Two sentences only if you're special ;-). Detailed analysis below.

The purpose, intent, and role of the queue are poorly defined, and as defined appear to be contradicted by implementation. The apparent goal of the submission queue is to be able to rate submissions for the site, and allow corrections of errors or problems with the submission. Changes are not currently possible.
Authors need to be able to edit submissions in queue. Very simply, problems should be fixable.
Authors need to be able to pull submissions in queue. Very simply, submissions which are fatally flawed, or replaced, should be killable.
As K5 grows, current posting algorithm means no submissions will post The current posting criteria operate in such a way that with sufficiently many users, no submissions are posted. The method needs to be changed.
Submissions hang too long in queue. This is both frustrating and unnecessary.
Queued submissions generate too much topical discussion. This obscures editorial content, skews moderation biases, and frustrates users whose posts "disappear".
Queued submissions generate too little editorial discussion.
Categorical feedback on submissions should exist. Checkbox items for typical requests, problems.
Feedback in the form of submissions posted / not posted should be provided. Worse than its failings (which are highly visible) is the fact that the queue's successes are invisible. The only time people are consciously aware of the sub queue is when it's broken.
Sectioning of submissions is erratic and often irrational. This reflects problems with both sections and the submission process.
Voting preferences vary greatly by section. Users of K5 have different interest. Users may have little or no interest in reading, or moderating, submissions in different sections of the site.
The best submissions are frequently dumped. Current voting algorithm only allows in very highly rated submissions. Debatable submissions languish, and eventually, die, usually taking much merited discussion down with them.
Moderation doesn't get used meaningfully. Other than a post/dump decision, a submission's queue performance matters little.
Diary entries are effectively ignored by the site. A method for promoting good diary material should exist.
Queue performance should impact authors. An author's queue submissions should measure against mojo.
Queue access should reflect past history. An author's access in queue should be partially driven by prior status and history, with possible limits on submissions in a time period.
A mechanism for author's comments is required. There's a place for context regarding a post which are neither comment nor submission.
Better moderation directions. Instructions to moderators should be reviewed and clarified.
The Queue is not a scheduling system. The submission queue has been used as a system for scheduling submissions. It's not.
Author profile should be expanded.
add item here.

[please don't resequence the above list, as the discussion page relies on keeping the same numbers]

Well, how about that. We've got an edit queue, and stories can be cancelled. Plus, with the 36-hour limit, stories don't hang around too long, or never get posted. Some other minor changes have been made, too.

I'm posting this on 9 May, a Thursday. If by next Friday, the 17th, nobody says anything, then I'll redo these three pages - update them with what's happened to Scoop since this site was first made. Not that anyone goes here anyway. :D

[Discussion of the above 20 points is on KuroshinSubmissionQueueDiscussion]

[A summary of suggested changes arising from that discussion is on KuroshinSubmissionQueueSuggestions]

CategoryKuroshin