Hey all, here is a conversation I had with Robin tonight and something I would like the community as a whole to be brought up to speed on:
- Hello Robin,
- There's an excellent reason why your pages weren't displaying on Familypedia for a few hours and that's because we had to turn SMW off there for about a period of three hours. We've been trying to work out a patch because Familypedia was causing massive load on our Apache and MySQL servers. The patch finally took and it was implemented immediately. Pages should be back to normal now.
- Robin, we're trying very hard to continue to facilitate your excellent wiki both because we do sincerely want to learn about and improve Semantic MediaWiki so other sites can be a storehouse of information like yours. But the template system you've implemented is insane (I mean that more in the good way than the crude way) and all of those queries and conditions and parameters that you get to handle of this information is simply overwhelming our DBs more and more as more and more content gets added. Frankly, if Familypedia were on its own server, it would have overwhelmed that stand-alone DB server a long time ago. It's a miracle our system has managed to handle this this long.
- In other words, there are simply too many blank pages. Too many expensive queries. Too heavily-coded templates. We're going to continue working on this on our end, but this is something you're going to /have/ to address on your end. Those templates need major reworking, even if it means cutting down on some of the functionality, in order for this to continue to work.
--daNASCAT (Help Forum) (blog) 04:47, November 13, 2010 (UTC)
It would be interesting to have some expert attention to what is generating the server load. What does "there are simply too many blank pages" mean?
In terms of our templates, is the problem in our "core" templates for displaying a person page, which queries the dates for the children, or is the problem the computation of "ancestry" and "descendants", where changing one page can effect thousands of other pages? It would be nice to have some instrumentation and logging to find out where the problem lies. Thurstan 07:17, November 13, 2010 (UTC
- I would "think" too many blank pages refers to Special:ShortPages, MANY of those could be merged into a parent page to reduce the number of pages the server has to index and work with. I really don't know what is going on with the need for so many separate Translations pages... Also it would probably help with the server load if the templates only asked for each piece of information once on a page. This could be done by storing the query in a variable either using the #arraymap function or something else such as the variable extension. SimAnt 18:49, November 19, 2010 (UTC)
- I don't "think" that "blank" is the same as "small", so I don't think that is the answer. These "small" pages are small because they only contain one template: they display a lot of text, so they cannot easily be merged back into their parent. We are still waiting for elucidation of what the new rules are, after the patch, so we have some idea how to fix all the pages which have been broken by the patch. Thurstan 20:45, November 19, 2010 (UTC)
- Only one query asking for the same thing on the same page. other formats such as format=count aren't restricted by this though. SimAnt 04:25, November 23, 2010 (UTC)
- Thank you for that: none of us had managed to guess it. Looks like we need something like the "variables" extension to cache attributes (like "sex") that are used several time on a page. Thurstan 19:56, November 23, 2010 (UTC)
I don't think it is working like that, which is causing a lot of our problems:
- {{#show: Edward Cook (1869-1919)|?birth_date}} returns "21 August 1869"
- {{#show: Edward Cook (1869-1919)|?death_date}} returns "7 August 1919"
so the second query for a different property of the same page is failing. Thurstan 20:58, November 25, 2010 (UTC)
- Yah, i noticed that two about #show and the exact same page being searched. I'm not sure what to do with that, I sent a Special:contact yesterday, since this is probably an oversight. SimAnt 21:36, November 25, 2010 (UTC)
Problems continue
When I try to display pages I get boxes with templates in them, like #show: See for example Joseph Doty (1651-c1732) This happens on every person page I look at, and no part of the page displays normally. DennisDoty 16:04, November 13, 2010 (UTC)
- Confirmed Joseph Doty (1651-c1732) as Dennis described - same as what all of them looked like yesterday. But a move back to normality is to "Edit with form" then just "Save". Page definitely recognizable; however, many box labels are links, and other aspects of the page are wrong (or maybe missing - I don't know what else is meant to be on that page). — Robin Patterson (Talk) 01:42, November 14, 2010 (UTC)
Patch
I cut the number of generations in the ancestor and descendant lists on the /sensor pages, and the number of generations on the /descendant pages.
As I have noted before, the trouble with the servers started when Thurstan began creating a large amount of /tree pages. rtol 14:26, November 16, 2010 (UTC)
Simplification
Note that I've started simplifying the main templates -- particularly, I'm removing the multilinguality and the links to forms. This improves the display, and makes it easier to read (and fix?) the templates.
Note also that I have very little time for this. rtol 22:11, November 25, 2010 (UTC)
- Thank you for what you manage to do, rtol. If we remove that part of our multilinguality, we should try to find a replacement. Phlox said he based ours on Commons:. — Robin Patterson (Talk) 22:27, November 26, 2010 (UTC)
- Removing some of the multilinguality has not restored the standard input form. Please see http://familypedia.wikia.com/wiki/User_talk:Pdlloyd#Biggest_problem_with_forms for Pamela's picture of the current form and comments on it and my response. Seems we need to replace all instances of {{String}} unless we can redirect them. — Robin Patterson (Talk) 02:48, November 30, 2010 (UTC)
- I think that is the story: {{String}} uses an SMW query of the same base page Strings:en, so it falls foul of the patch. Thurstan 03:18, November 30, 2010 (UTC)
Are we waiting on wikia or on ourselves?
It doesn't seem that there's been progress in a while - form-derived pages still don't display properly, and the (English) labels don't appear on the form page. Are we still waiting for the Wikia staff to fully re-enable SMW, or have they already done so? If the latter, do we know what needs to be done to counteract the side effects of the template modifications that were supposed to reduce server demand? Bruce Kendall 20:50, December 12, 2010 (UTC)