Question Regarding the built-in Posts-Per-Page Limit and Where it Comes From

xrunner

  • Posts: 192
I had worked on an idea today regarding forum "pages" and a different way to go through a thread's posts, but after reading what Mr. Arantor and Mr. Nao had alluded to in a PM, I need to ask a theoretical question now. If my idea doesn't work out and I just learn something, so much the better. I'd like to ask about where the limitations of how many posts per page comes from.

By process of elimination I found that the SMF "Number of posts per page in a topic page" would accept a number up to as high as 999. Entering 1000 made it revert to 15. Never even tried numbers that high before and never heard anyone talk about it much. On the surface it sounds like one of those "nice round numbers" somebody decided to use, but it may have come about by some sort of objective testing in the past.

I found the largest thread by replies on my forum that had 157 pages called "word association". My posts per page setting was 25, so that's about 3925 posts. Being brave, I set my active forum's posts per page to 999, and after a few seconds it came back with four pages, having re-sorted it all correctly. One hell of a scroll though (that in and of itself would not have impacted my idea though).

On another forum I can get into, I found the largest thread by replies was 67 pages @ 29 posts per page for a total of ~1943 posts, so the setting could have been set much higher there.

But on a much bigger forum redandwhitekop the largest topic by replies was 1569 pages. Don't know what sort on impact a high setting would have there.

Anyway, take the built-in limitation of 999 posts per page for SMF. That's a pretty high allowable limit as it is. What thinking lies behind that number? Is it a limitation for the server, or a limitation for the outside viewers or spiders?

Thank you.

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
999 is mostly a safety barrier.

On all levels, it's a balance between performance and usability. Too many on one page is a performance killer, because it takes longer to process, plus eats more bandwidth - but it would use less bandwidth than each page being requested separately, of course.

As Ben (RAWK's owner) found, there is actually a much finer balance under the hood - more pages = fewer posts per page = more queries to be issued = harder hit in the long run. Plus there are all sorts of implied limitations in terms of using more resources and limiting others.

The 999 then is more a theoretical rather than practical limit, and 20-40 per page is really a more practical limit, ideally 15-20 for smaller forums and 20-25 for larger forums as that's what's shown to work best in terms of balancing requests (and all the queries with that) with the load of processing each request.

There is also the implication attached to infinite scrolling and all the same issues apply there too (just you don't explicitly click on the next page button)... plus all the attendant issues with too much content on mobile and of course search engines - they don't do infinite scrolling, and too much content in a page is bad for SEO too.
When we unite against a common enemy that attacks our ethos, it nurtures group solidarity. Trolls are sensational, yes, but we keep everyone honest. | Game Memorial

xrunner

  • Posts: 192
I see. Thanks.

The idea I was tinkering with, was to consider the entire thread, no matter if it was 10 posts or 1000, as an entity that was browsable in terms of a percentage - not a "page". I don't think people care about pages per se, they sometimes go across long threads to see how the progression went. They want to perhaps go to the 50% point, or the 75% point, and so on. To my mind, talking about pages or dividing a thread into pages is not really a help to people. That's the way it's been done, but I wanted to come up with a way to scroll across the totality of posts and the indicator was to be a percent indicator of where you were with respect to the total posts in the thread - not a "page" number.

But I ran into computational concerns ... well if this spurs any ideas then there ya go. :)

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
People might not care about pages, per se, though they do like to have 'units of content', especially to be able to refer to.

There are a lot of issues in terms of scrolling through the totality of a thread - namely that once it reaches a critical mass, it's simply too big to consider the entire thread history. Once you get beyond a couple of hundred posts, generally, it's too big to keep on track properly - so you don't. Pages don't actually have too much of a problem in that, actually because '50%' isn't a big deal. You only really need to worry about getting 'roughly' right. In a 100 page thread, when you're looking for half way, do you really care whether it's page 49 or 50 or 51 as being 'half way'? You just want to get a handle on it.

Sure, we could display a 'you are |---------------|____| this far' through the topic. It might not be a bad idea to experiment with as a plugin actually.

But the thing is, we're slowly being conditioned to phase out paginated content - and there are plenty of fantastic reasons to keep that (coming back to where you were previously, search engines). Consider Facebook, consider Twitter... if it's a few hours old, it's off your page and might as well be gone[1] - the totality of time there does sort of mean something and FB Timeline did sort of give us some idea of that, that waaaaaaaaaaaaay back something happened, then you joined Facebook and OMG ISN'T IT SO EXCITING HOW MUCH STUFF EVERYONE DID. No, no it isn't.

Infinite scroll is one of those slightly quirky things. It's an interesting feature and it certainly has a place in reading long discussions. But it's certainly not the only methodology, it's certainly not mandatory to use it (unlike FB/Twitter) - and it has a place too.

Would you read a book if you didn't have 'pages' but simply one long thing to scroll through?[2]
 1. Substitute days if you're boring like me and don't have many friends on FB/Twitter because you find most of what most people post is fairly dull and uninteresting. That's not that I know boring people. It's just that they open the taps and post everything, no matter how boring it might be. You'll find forums don't have, generally, the same effluent rate that FB/Twitter does.
 2. I have several e-books that are single-HTML pages. I don't recall ever actually scanning the entirety at any time.

xrunner

  • Posts: 192
Quote from Arantor on June 10th, 2013, 02:07 AM
Sure, we could display a 'you are |---------------|____| this far' through the topic. It might not be a bad idea to experiment with as a plugin actually.
That's nearly exactly the text representation I had on the file on my other computer (which I gave up on for the time being). :lol:

I had all sorts of ways to compute the percentage where you were and where you wanted to go in the thread, but I ended up thinking it would have been so responsive and so dynamic, that it just wouldn't fly on a real forum. :-/
Quote
Would you read a book if you didn't have 'pages' but simply one long thing to scroll through?
I thought about this for a while. Now, I have had 1 and 1/2 glasses of wine, so take that into consideration ...

I think so, sure. As long as I can mark what page I left off reading, and if there was a table of contents that would point to at least a symbol on certain pages (which was one argument I wanted to make against pages on a forum - what are pages without a table of contents :wow:)

So I say - yea I would read a book without page numbers. Now I suspect you will try, and succeed, to cause me to reverse my claim. ::)


Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Computing the percentage is relatively easy when you get down to it. You know the number of posts in the thread, you can figure out where the user is readily enough, so you know where you are in terms of the whole. Where you are divided by the total multiplied by the size of the bar = how much of the bar is filled in. Doing it dynamically isn't really much of a challenge in practical terms.
Quote
I think so, sure. As long as I can mark what page I left off reading, and if there was a table of contents that would point to at least a symbol on certain pages (which was one argument I wanted to make against pages on a forum - what are pages without a table of contents :wow:)

So I say - yea I would read a book without page numbers. Now I suspect you will try, and succeed, to cause me to reverse my claim. ::)
Oh, I wasn't trying to reverse the claim. But I will undermine it somewhat.

So, let's take the comparison fairly: one chapter of a book is one topic.[1] You're using symbols to indicate where you are for your own reference. It doesn't matter what that symbol is as long as you can recognise it. Some people use bookmarks, some turn the corner or some other physical marker. Whatever, you have a marker for your use to indicate where you are.

And if that was all it was - a reference point for you and you alone, the argument would stand quite happily. But a forum is implicitly not a private resource. The markers are not entirely for you. You will still have your own markers, of course, but a forum is a shared resource. Other people will want their markers too.

Since we're already doing markers, let's be really daring and set out markers that everyone can use. And just for shits 'n' giggles, let's use those funny Arabic squiggles... Oh wait... now I have page numbers.

Putting aside the whole 'everyone can set their own page numbers'[2] if you have a common reference, you have a common way to gauge the length of a topic in terms of relevance (a 10 page topic is more likely to be heated and/or drifty than a 2 page topic), that everyone can agree on and see the same thing.

It then means you can say 'jeebus, that went on for 3 damn pages' and everyone knows what you mean.

Then you have the argument of bookmarking - but sharing that bookmark. Say you find a particularly thrilling paragraph, you can't just say 'about 1/2 way down and if you get to <that bit> you've missed it'. But if you have a common frame of reference, you can direct people to it. Sure, we have post links and permalinks and whatnot but it sort of misses some context to be able to point to the page as a whole. And that goes for sharing.

Then there's the psychological aspect. Humans don't do well with vast amounts of stuff. We're born to filter out and prioritise[3] and that means we do a process frequently referred to as chunking. We take a given process or data set and process it into a more manageable form so we don't have to do it again. Giving a standardised chunk of content, we'll do that. We'll mentally scan and summarise - and store the summary thereof. Having pages is a convenient enabler for that process. Especially in a discussion with multiple parties and whatnot. And of course if the thread is very long, you might just read the first post and skip to second-to-last page.

The long story short... yes, you would likely read a book without page numbers. But it's not a book that everyone else can read and write the next chapter to at the same time. A forum on the other hand... :lol:
 1. I have successfully co-authored a novel using the forum as a medium for doing so.
 2. Which most people never do, and most forum systems don't have the option anyway. I should really remove it in Wedge, it serves little real benefit.
 3. cf. all the studies about looking at pictures of planes and crowds and not noticing that an engine has disappeared, or the gorilla walking through the crowd or whatever.

xrunner

  • Posts: 192
That's very good material for consideration. Some of it I had stumbled upon today on my own - that's why I didn't post my idea in full because I realized there were flaws, and that's why I needed to ask some questions. :)

But, I have some conflicts about all this still swirling in my head [1]. When I devoted some time to this today I was, all the time thinking, "Surely some ultra-intelligent person has already gone through this". Yet, I'm still troubled by a few concepts relating to why "pages" still have to be used on forums. I mean today the Admin can say there are 50 posts to a page, and tomorrow he/she can say there are only 25 ...

Maybe I'll resolve these conflicts tonight or tomorrow. Maybe I won't. Let me consider what you have said, with the ideas I had in my other text file. Regardless, it's an interesting and instructive hashing-out of ideas. :cool:

 1. feeble as it is

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
You could but it would be flawed.

In the case of a YouTube video, very often an individual frame may not have much substance in itself and it may not work out of context on its own. So too with posts of a thread, they may have little substance, or they may have everything... and out of context they may not work well.

The difference is, though, generally a video is expected to be considered in entirety, while a thread not so much. For long threads, commenting upon them is fine in reference to the last few posts/last page or two is expected, rather than reading the entire thread. Contextually we're talking about a jump-cut to another scene or location under some circumstances. It still relates to the whole but it might have a twist to it that requires new context.

xrunner

  • Posts: 192
Quote from Arantor on June 12th, 2013, 01:05 AM
You could but it would be flawed.
Most probably, but it's fun hashing these ideas out with you. :) [1]
Quote
In the case of a YouTube video, very often an individual frame may not have much substance in itself and it may not work out of context on its own. So too with posts of a thread, they may have little substance, or they may have everything... and out of context they may not work well.
I agree. Many posts in a thread have little substance too, especially longer ones where people just post "LOL" or other noise. So the analogy sortof holds so far ...
Quote
The difference is, though, generally a video is expected to be considered in entirety, while a thread not so much.
Well, I have scrolled through videos for a specific part that I was interested in and ignored the rest. I've done the same with a thread.
Quote
For long threads, commenting upon them is fine in reference to the last few posts/last page or two is expected, rather than reading the entire thread. Contextually we're talking about a jump-cut to another scene or location under some circumstances. It still relates to the whole but it might have a twist to it that requires new context.
What I was going for as I said was each video is analogous to a thread, and each frame of video analogous to a post in the thread. In the video, you can (and I do quite often) scroll though it to the information I want to get out of it. For example, I value the posts of certain members over other. I might scroll through the thread to just read what I consider the "smart" people are posting. In videos, I can scroll the bar and look for what a certain person is doing without regard to anything else. In threads, you can do the same thing, especially if they are long threads by looking to members names that you want to follow.

Yet, in the video, there are no "pages" of frames - it's just the whole video from start to stop. Nobody cares about the number of frames or that they aren't divided into "pages" of frames - they care about what they can find in the video. I'm just wondering how people would react to a thread that could be "scrolled" from beginning to end, as a video, without regard to "pages".

Now, to destroy my own argument, I'd probably say ... ... because each post has information in it that needs to be able to be searched, whereas a frame of video, which does have information, is (as far as I know) impossible to search at this point in time, all posts have to exist all at once in bins "pages" for search engines. :hmm:
 1. at least I get fun out of it - thanks

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Oh, I saw exactly where you were going with this. *cracks knuckles*
Quote
What I was going for as I said was each video is analogous to a thread
No, it isn't. The nature of the medium inherently precludes it being the case.

You don't have films made frame by frame by (mostly) different people frame to frame. Each frame directly follows on from the previous one in the *vast* majority of cases, and where it does not (i.e. with a jump cut or similar style of cutaway), that could reasonably be concluded to be the end of a scene. Which is the correct analogue for a post, really.

OK, let's get complicated :niark:

What is the smallest unit within the frame of reference? Within a video, you have chapters (in most longer works, e.g. DVD movies), which comprise of scenes, which compose of frames. A frame is the smallest unit to which you can meaningfully refer.

What of threads? Well, you have a topic for collecting posts, you have pages of posts, which compose of sentences. A sentence, then, is the smallest unit to which you can meaningfully refer.

You can base an entire new reply off a single sentence or even one word in that sentence. You can base an entire new scene off one frame or even one element in that frame.

A frame, at best, compares to a paragraph or a sentence. Given that point of reference, the concept of pages of posts is no more radical than chapters on a DVD menu; a way of collating a group of posts together in a unit that is not overly large and complex to process.

Put it this way, you might pause through each frame to study it and gain context and meta information from it but it won't tell you anything about the whole, just as you can step through each sentence to do the same with the same caveats.

The only difference is that the DVD menu author doesn't have an automatic splitting point for it, they must manually do it themselves when authoring the menu. Maybe there's an argument there for having an ability to semantically group posts together in accordance with some criteria that isn't a mostly-arbitrary number. Perhaps the angle to consider is somehow dividing a topic into pages, where each page is a variable length exploring one aspect of the thread as a whole.

There are practical implications for so doing; the main one being that it is inherent resistance to discussion that most people won't want. People are not particularly good at categorisation especially when it is not for their own inherent benefit; cf. why most forums with tagging systems seldom make good use of them. Moderators might if they're enthusiastic enough at it but most topic starters just won't bother unless it gives them some benefit; for example on Stack Overflow, when asking a question, appropriate tags can be set. This is directly useful to the immediate author as well as those who would come after, for the simple reason that people actively follow certain tags and if a given question has a given tag, it increases the likelihood that someone knowledgeable will see it. (= topic starter has an explicit benefit to themselves as well as an implicit benefit to everyone else)

xrunner

  • Posts: 192
Quote from Arantor on June 12th, 2013, 01:58 AM
Oh, I saw exactly where you were going with this. *cracks knuckles*
I know. :)

Let me consider what you have written until tomorrow. :hmm:

But I will respond with a question that kindof fits in, in one sense.
Quote
OK, let's get complicated :niark:
^^^ See that smiley you used? I use them too.

But, I had one moderator at another forum that said (paraphrasing) that he would never resort to using smileys. He claimed that, if you were intelligent enough, you wouldn't have to resort to using them. If you were intelligent enough, you could explain yourself without using a smiley.

Hmmm.

I don't really quite agree with that. It seems to me that a smiley is akin to a facial expression when people are in front of each other. It's part of the way humans communicate without words, which I think is very much needed on a forum.

Oh, I just got a new idea regarding smileys! I swear I just now thought of it Ha!

But I'll wait to hear what you say about smileys first. Are they "cheating" or being lazy when used in a forum conversation?

Arantor

  • As powerful as possible, as complex as necessary.
  • Posts: 14,278
Quote
But, I had one moderator at another forum that said (paraphrasing) that he would never resort to using smileys. He claimed that, if you were intelligent enough, you wouldn't have to resort to using them. If you were intelligent enough, you could explain yourself without using a smiley.
I can see the logic of that statement, but your moderator is wrong. Go ask any counsellor, psychologist or similar communications specialist how much communication is conducted with the spoken word. The estimate will vary from teaching to teaching but the highest I've *ever* heard it quoted in 25%...  and I've heard it quoted down to 7%. In other words, 75% or more of communication is not what you're saying or even really how you're saying it. It's all the non-verbal stuff too.

But of course we don't have that additional context in a forum that is primarily textual. You're cutting off that 75%+ of context. It's even less context that you get in a phone call when you at least have tone of voice to guide you in terms of stress on the part of the speaker. All you ultimately take away is what I write.

Now, I would argue that I am a reasonably strong communicator; I have sufficient command of English - generally[1] - to be able to make my point and cover discussion in depth (when I choose to), and I tend to choose my wording fairly carefully to convey the meaning I intended, however most of that is a subtlety often lost in this medium. Sarcasm in particular doesn't tend to translate too well.[2]

Ultimately, to me, it's not cheating, it's a way to express a hint of the missing context that goes with non-verbal communication. The tone of voice, the expression on the face... with a suitably well defined smiley set, a surprising amount of additional context can be easily communicated.
 1. On a similar line of thought, I have heard it said that you shouldn't need to resort to swearing if you have a sufficient command of language. I find I resort to swearing when more erudite communication would be wasted or simply that I cannot find the words to express my frustration. There is something very earthy and primal about profanity to express level of frustration.
 2. But you will notice that I choose not to act like a grammar pedant or express pedantry over punctuation. But I see a lot of dumbness, like the guy who complained that the Cmdr Hadfield singing Space Oddity while on the ISS must be fake because sound doesn't travel through a vacuum. Or the folks who say that the Sun isn't a star, it's a Sun.

xrunner

  • Posts: 192
Quote from Arantor on June 12th, 2013, 02:28 AM
Now, I would argue that I am a reasonably strong communicator;
:yahoo:
[1]
Quote
I have sufficient command of English - generally[2] - to be able to make my point and cover discussion in depth (when I choose to), and I tend to choose my wording fairly carefully to convey the meaning I intended, however most of that is a subtlety often lost in this medium.
 2. On a similar line of thought, I have heard it said that you shouldn't need to resort to swearing if you have a sufficient command of language. I find I resort to swearing when more erudite communication would be wasted or simply that I cannot find the words to express my frustration. There is something very earthy and primal about profanity to express level of frustration.
Oh I agree 100% I don't swear often on a forum, but if I choose the right time and place, one swear word works very well, thank you.
Quote
Ultimately, to me, it's not cheating, it's a way to express a hint of the missing context that goes with non-verbal communication. The tone of voice, the expression on the face... with a suitably well defined smiley set, a surprising amount of additional context can be easily communicated.
Compare the difference between these two statements (on a forum of course). Lets' say Wedge is released for general use and a newbie asks a dumb question in the inevitable support forum, and you respond in a way the newbie doesn't seem to appreciate.

I post this to the newbie -

"Oh yes, Arantor is a horrible communicator."

That's basically a straightforward assertion, to be taken without further interpretation.

But if I post this -

"Oh yes, Arantor is a horrible communicator" :) (a simple smiling face)

It's fairly obvious that there's more going on than just the words convey. So yes smileys do add something that can't quite be added by more words.
 1. Note appropriate use of non-verbal communications