Forum:SMW/Archive 2009-07

Semantic mediawiki status: Familypedia support is "pre-alpha". (Experimentation is welcome, but there may be radical changes). Users need not be concerned that their contributions will be wasted, since their articles will eventually be upgraded to the new format without loss of data.

Summary of where we are at

 * copied from a user talk page

Currently, we are revising the way that we centrally store information, so everything you are learning about info pages and showinfo etc will be still work and all- it just will be obsolete. ... We will be encouraging people to use the new mechanism and be converting data over for them.

We are using semantic mediawiki extensions, something I believe that WP will eventually have. The new templates and properties are not yet ready for prime time, but if you are curious, you can see them demonstrated in article George Spencer Geer (1836) instead of the error prone process of editing info pages (a mechanism of my creation), the new scheme uses forms. See the edit with form item in the menu bar. This stores information directly in MySQL relations so that we can do databasey things like search for people born in location, with birth date greater than somedate and less than other date. This sort of thing is partly activated due to a hack I put into the info page mechanism. Those pages go away in the future, and pages more resemble those of the geer article. - ~  Ph l o x  01:21, 1 June 2009 (UTC)

Even clearer indication from Phlox

 * (Extracts from recent conversation on another forum)


 * What I and other "followers" would like is an assurance that the info put into info pages will not be wasted but can be converted to SMW facts etc by a bot or similar automatic process. — Robin Patterson (Talk) 02:37, 14 June 2009 (UTC)

Nothing will be wasted, and info pages have given us a huge head start because the data is directly transferable. ... everything will be transferred over from info pages. ... Mind you, some of the info page values have been used in odd ways, such as place names in date fields. People ... will really want to correct these problems to get the most out of the SMW features.

William, I think you will see/find that form entry is vastly superior to the error-prone info pages method. And the volume and detail of information one can enter has been increased as well. What is particularly attractive is that when you are entering place names, the Familypedia will autocomplete the entry for you. So if a county name is in 3 different states, you will be made aware of that as you are typing in. This autocompletion is also active for all file and article fields, so there is no problem with typos as with info pages. '''Some of this stuff is working and you can create your own articles using Form:Person. People are welcome to try creating their own articles using it.''' Understand that it is under construction and it is not ready for prime time yet... . Certainly if anyone would like to stay with the info pages, they are welcome to. The SMW can coexist perfectly well with info pages. Of course, contributors making that choice will not be able to access any of the new features such as a rich set of database queries, and dynamic categories. For example, one could have a table on one's Userpage that dynamically displays all changes to articles for all surnames the contributor is interested in since the last time the contributor logged in. I think everyone will be pleased with the features that SMW will deliver for Familypedia. - ~  Ph l o x  04:45, 14 June 2009 (UTC)


 * I intend to start testing the forms on new Person articles soon. Will the conversion from info pages be done automatically at some time in the near future and will contributors who wish to stay with info pages (not me right now - I want to check out the forms first) be able to indicate that they don't want their articles converted? Bill Hunsicker 05:27, 15 June 2009 (UTC)


 * As Familypedia policy regarding bots, all site wide automated conversions (this one included) take place only after there has been adequate public notice to the community and there has been opportunity for objections to be raised. To your question, the answer is yes- Contributors will have the opportunity to specify articles they have been the primary contributor on to opt out of this conversion phase.  I shall provide details when we are closer to the conversion phase.  Although a bot run could easily transfer all info pages in a few hours, the conversion will be gradual, rather than a single massive push.  This will allow us to notice any shortcomings in the conversion, and more importantly, the SMW structures that they are being converted into.
 * SMW is not in beta test yet: Bill, I am glad you realize that this should be used for testing only at this point. The Form:Person may look polished, and the query pages may deliver interesting results but believe me, the SMW pages are very much a construction site with heavy equipment slamming into structures every now and then. If to fix something I need to rename some templates or parameters, that means any SMW test articles may partially break.  For example, as thurstan noted last night, birth-state was recently renamed to birth-subdivision1.  Well, that sort of thing is trivial to fix in test articles (just renaming a parameter in an article), but I don't have the time to go back and fix test articles other than the ones I use, so you are on your own until SMW goes into beta test.  We simply aren't there yet, and if you want to use it as your primary input means that is fine, but understand that you may have to go back and fixup articles, so it is not for the faint of heart. There are enormous numbers of things to tweak, but at this point what we need to understand is if there are any major structural errors- like missing the capture of some information, how we are doing events, or multilingual, and so on.
 * Regarding the timing of the conversion, I am in no big hurry and shall remain reluctant to do this until the structure is more tested and mature. I would have liked us to have a more complete geographic knowlegebase prior to the conversion, but I suppose we could do it as a post process cleanup by marking the origin of the smw pages and later going back to translate geographic names more properly.  For instance, Greene county ->Greene County, Alabama rather than Greene County, Arkansas.  In any case, we will need this database prior to going into beta, because while autocompletion sort of works now, it is using an unclean database that uses categories were never intended for this purpose- you get suggestions like map of XX county for a county name autocompletion.  This geographic knowlegebase bot run shall not be modest and the main purpose is to generate autocompletion lists. A byproduct will be substantially more of the auxilliary articles on places.  I figure we should have all villages and towns mentioned in wikipedia, not to mention the counties and subdivision1's.
 * - ~  Ph l o x  17:41, 15 June 2009 (UTC)


 * I have created one test Person page (Abraham Hunsberger (1786-1860)) and although the page looks sparse compared to the info page I like the layout of what is there. I also created a Biography Template that works with the facts on this page. Thank you for your assistance with that. I think that I will create another page with a Person who has a family to see what that looks like. Otherwise, I will be patiently awaiting developments. Bill Hunsicker 21:02, 15 June 2009 (UTC)

More wonders
I'm beginning to get the hang of this. See Descendants of Charlemagne (couples) for another demonstration of the powers of SMW. The same structure can be used to construct lists of, say, Franco-Italian couples. rtol 09:42, 14 June 2009 (UTC)

Listing in birth order
Rtol will see that I have indeed looked at the abovementioned Descendants of Charlemagne (couples). It lists children in birth order: great; but does that risk omitting some, or has that problem been fixed? — Robin Patterson (Talk) 14:07, 14 June 2009 (UTC)


 * Children in birth order works fine. There are problems with the first column only. rtol 18:32, 14 June 2009 (UTC)

We are not in Beta test mode yet
Observations on cosmetic improvements of course are are welcome, but the attention at this point is on isolating any required improvements to the structure of the data we are capturing.

Note also that Form:Person very intentionally at this point does not meet familypedia's requirements we had for the Person page and Simple page. A simple form will subsequently be derived from this exhaustive presentation. So folks shouldn't be alarmed that I am cramming a kitchen sink of properties that people may only rarely fill out into the Person form. (Just wait until I get to the genetic (YSTR mitochondrial DNA) subform- I'll bet we get only a few takers in the near term on that one). The goal is to push the envelope and specify classes of properties that potentially could require rethinking the basic structures. At some point when things are stable, we will present the simple forms as a default. The advanced/ expert mode forms will probably be implemented as something that appears when a CSS property is set- like how I showed Fred how to do red bars. People would copy a little value into their monaco.css and voila, they would always be presented expert forms rather than have to always click a link to get them. - ~  <font color="#0DC4F2">Ph <font color="#3DD0F5">l <font color="#6EDCF7">o <font color="#9EE8FA">x  19:11, 15 June 2009 (UTC)

Transition plans: some proposals
(For background, see User talk:Phlox).

At present the template Set facts from info/family, which puts the list of children into an SMW property only handles the first 9 children in each group. This is unsatisfactory, because it means that all the planned features which depend on properties Ahnentafel and Descendants will not work reliably across big families. On the other hand, we do not want to increase server load unduly. For a "normal" person page with an "/info" page, the present code retrieves the children twice: once for display, and once to load the properties. One solution is:

Proposal 1: Take the "children" code out of Set facts from info/family, and modify Showinfo children so that it sets the property for each child as they are displayed.

Discussion: there would need to be a flag to make the "property setting" conditional, since Showinfo children is occasionally used to "show siblings". Since this is the much less common case, I propose that the default value for the flag should be "on". Note that (possibly at the expense of another flag) this change could also fix the case of the children being stored on the spouse's /info page.

Once we have started on this line of thought, we realize that Set facts from info/family retrieves other values which have already been displayed, so we have:

Proposal 2: Modify Showinfo person so that it sets (most) of the values that is retrieves into the corresponding properties, leaving Set facts from info/family to handle just the values that are not displayed.

You may take it as read that I am volunteering to implement these proposals.

Thurstan 03:35, 16 June 2009 (UTC)
 * Cool idea. You understand the need to minimize wikia server load, so I trust your judgement.  One request: When you come across a field like baptism that could have both a date and a location, please stick it in street-address.  They should display fine in showfacts person and we will automatically know that any values in those fields are likely from xferred infopages even if the marker category is dropped, and be able to post process them with bots. - <font color="#0A9DC2">~  <font color="#0DC4F2">Ph <font color="#3DD0F5">l <font color="#6EDCF7">o <font color="#9EE8FA">x   03:59, 16 June 2009 (UTC)


 * Okay, though that's really "Proposal 2": I'll do the children first. Thurstan 04:14, 16 June 2009 (UTC)


 * Well, I have done "Proposal 1" (documentation still to be done) . I will wait for the dust to settle before doing anything more. Thurstan 06:57, 16 June 2009 (UTC)


 * "Showinfo children" now assigns properties. Unfortunately, "showinfo children" is also used to show siblings (even though there is a template "showinfo siblings") so that siblings (and self) are listed as children and hence as descendants. Have a look at Floris I van Holland (1030-1061) who is his own father, grand-father, great-grand-father ... Amazing that the system has not blown up yet. 05:25, 18 June 2009 (UTC)


 * I foresaw that: you have to code to show siblings (and similarly in any other context where you don't want to set properties, like on documentation pages). The problem is, it is not easy to find instances where this is needed. Can we frame an SMW query to find the people who have become their own children? Thurstan 05:57, 18 June 2009 (UTC)


 * Noted. I don't think this is a robust solution. We should just use Showinfo siblings if we want to show siblings.
 * Show ancestors has a basic query on the descendants list. rtol 06:00, 18 June 2009 (UTC)


 * Trouble with Showinfo siblings was that its clever inventor forgot to tell the rest of us it existed. It is uncategorized and is linked from only one of his templates and a dozen individuals' pages, mostly Dutch. I'll substitute it on the info article template. — Robin Patterson (Talk) 11:41, 19 June 2009 (UTC)


 * A search on "" finds well over a 500 pages. Rather than changing all of these, I would suggest that we change the template showinfo children back to what it what. info categories already adequately turns the children into properties anyway. rtol 17:05, 25 June 2009 (UTC)

More thoughts on "where are we going?"
I notice a recent remark on converting yewenyi's pages [I can't find it again just now: where did I see it?] got me thinking about "what are we going to do with all the existing pages that use "old" styles?". What is our "vision" for the "SMW" future? Will all pages use the "set facts" templates? See User_talk:Thurstan for a contributor asking me not to convert "his" pages to (my) "standard" form. I think the developers of SMW pictured a very simple usage which didn't involve lots of templates, and I think we could use it like that on some of the "old" pages: see what I have done with Alfred Edward Latter (1868-1952)‎. We could use this style on any pages using childbox and the old "simple person" model, and any other format that results in a "list of facts" (either in an infobox, or in the main text). Thurstan 03:58, 17 June 2009 (UTC)
 * Actually I marked up just such an article a month ago in the same way. This led me to simplify the naming of the properties so that they would be easier for such inline use background here.  SMW doesn't care whether we use forms or not, or whether we even use templates.  The "facts" properties can be used perfectly fine without a single template, and if lots of folks get excited about doing it that way, I won't argue with them.  You of course are welcome to document that method, but I am skeptical of the reception.  Philosophically, the SMW guys think in the same mode as the microformats crowd, that if you keep metadata inline with regular text that people will keep the information up to date because they are always looking at it.  Wikipedia has an audience of contributors that have no trouble with wikitext and SMW like nerdy markup.  Should we expect that the audience of folks interested in genealogies and family history will get over the Frankenstein reaction to text junked up with weird wikitext syntax and even weirder SMW property names?  They may well wonder why they should bother putting any of those square brackets around dates or correctly labeling one as death date as opposed to the remains date.  The inline thing syntax displays the date just fine.  Also, they can make the date appear as they want.  As with any square bracket syntax, the right hand operand is the format that is displayed.  THey may not understand that SMW allows dates to be formatted according to the user's preferences, but only if the contributor does not use the right hand operand for display text.  So ok- maybe they favor month first, or day first, or maybe they really really like YY/MM/DD format.  But what the heck- it's a free world, right?  Most folks will never get anywhere near this comfortable with wiki syntax and find wikitext baffling.  The fact is, we have trouble attracting people to wikia for the very reason that even syntax that is much more simple than SMW declarations puts them off.  SMW authors may run in a crowd of folks that have a high tolerance for wikitext like syntax, but typical folks do not.  That is why wikia is going in the opposite direction- introducing an editor that further hides the complexity of such syntax.  There is not much harm in offering thhis mode of entry, but we should be realistic. SMW syntax is not pretty and we can't dream that we can appeal only to folks comfortable with double colons and square brackets.


 * The question "What are we going to do with all the "old styles" has been asked in various forms a long time before SMW appeared. Implicit in some people's phrasing of the question is that maybe it is a good idea from a kind of egalitarian point of view to have lots and lots of article styles.  At the end of the day, this question of the multiple styles has nothing to do with SMW or any other technology for that matter.  It has to do with polish- a common look and feel.  From an editorial perspective, no other wikia of our size has anywhere near as disorganized a presentation of information, with such a bewildering mishmash of looks to the articles.
 * My opinion is that the sooner Familypedia makes up its mind about a standard look and feel to articles, the better. I can sympathize that some folks will want to keep their little fiefdoms in one particular format.  Well whatever- if 80% of our content would adhere to a common look, I think we will have made a great leap forward in establishing this as a quality site.  This is the way wikipedia has gone, and it makes sense to settle on one common styleguide.  - <font color="#0A9DC2">~  <font color="#0DC4F2">Ph <font color="#3DD0F5">l <font color="#6EDCF7">o <font color="#9EE8FA">x   07:08, 17 June 2009 (UTC)


 * Thank you for your thoughts. I would like there to be a "standard" style, since I do often fiddle with "other people's" pages, and it would be easier if they all had a common infrastructure for the metadata (I know that this rather contradicts what I said above, but we can't all be consistent all the time!). Thurstan 07:22, 17 June 2009 (UTC)
 * Our edits crossed- there were updates to my prior note. It's a delicate line.  You want folks contributing, so you don't want to irritate them by telling them they can't use the clever floral borders that really do look very professional and truly set their articles apart from those of others.  Then there are guys that have been doing this for decades and you can't tell them anything about how to do a table with genealogy information.  They made up their minds about that years ago and they will do it their way come hell or high water.  Genealogy folks are an odd breed- I suppose Robin oftentimes feels like he is herding cats.  It takes tack and diplomacy, something that the good father did not issue me in great quantities, so I am glad others are around to move us along towards a common standard. - <font color="#0A9DC2">~  <font color="#0DC4F2">Ph <font color="#3DD0F5">l <font color="#6EDCF7">o <font color="#9EE8FA">x   07:50, 17 June 2009 (UTC)

Standardizing page look
I agree with you chaps on the whole. And AMK152 was also keen on reducing the variety. Our only easily-found "simple" model includes in its introduction a suggestion that users might save time by choosing info pages (with a hint of the form/fact future). One move that might be easy is to see whether the "simple" page could be tweaked to look even more like the newer standard and maybe to vary its input parameters if they differ from the modern ones. That might facilitate later upgrading. — Robin Patterson (Talk) 14:44, 17 June 2009 (UTC)


 * I, too, would like to see a standardized page look that can be easily modified to give contributors a feeling that they can "have it their way" while keeping things within a broad standard. Visitors will be confused and possibly put off if information is in radically different places on different pages. I have no strict preferences on page look as long as there is some uniformity so that visitors can easily find the information they are looking for. The main purpose of the Biography templates is to streamline the process of putting some kind of narrative on each of the pages with minimal effort. I would be happy to see ideas and improvements to the Biography templates to make them as great a value for Familypedia as possible. Bill Hunsicker 02:01, 20 June 2009 (UTC)

Feedback on Robin's father-in-law's page
See Talk:David Brian Carrad (1916-2003) for some notes about progress. — Robin Patterson (Talk) 01:03, 19 June 2009 (UTC)

The Denobulan Gazette
Here's the main news for the past few days:
 * A method was discovered for reliably allowing wikitext in a property. Prior schemes would lock up the page when particular templates were used.  This allows Description, Source and Notes properties to use standard wiki links and other markup.  Note that complex templates like Citation may become translated internally into a subst'd form.  I am not doing this- the Semantic Forms engine is doing it in order to protect SMW from potentially disruptive wikitext in properties.  However, the value in the form retains its original form and may be edited normally.  Use templates with caution and report any tricks or unusual problems to me.  Citation in particular is very valuable for serious genealogists and use of its various forms should be encouraged in the Familypedia Manual of Style.
 * The new page editor's interaction with typical Familypedia pages was investigated. The first huge problem is that we have large numbers of pages with embedded comments, and the new editor does not like them.  I describe a method of using dummy templates in place of embedded comments at wikia central, but the developers may have support for comments real soon now.  In the meantime, all our standard template articles can't be edited in the new editor and default to the old.  See other observations esp regarding their informal forms at wikia central.
 * Some demonstration work was done with drilldown filters. See Drilldown example for the trial articles.  I imagine our filtering will be more along the lines of the dynamically created gallery at The_Hague/1900-1919 or query tables either static on a page, or generated from the query page like this one that is linked to via Births nav link in The Hague article.  This is a showcase article for locations, similar to the showcase articles I use for persons- George Spencer Geer (1836) or Henry I, King of England (1068-1135) for stress tests and multilingual cases.  It is not as fully developed as the person articles have taken precedence.
 * PhloxBot did a run correcting a number of errors in the trial articles:
 * Non existent parameters were deleted. There is no "Day" parameter.  It is part of "date"
 * Date parameters did not use the proper format. Script, Bot and template contributors may paste the following pattern for accurate translation from a wide variety of input formats.  For instance, 12 May 1847 converts to the required "1847/05/12".
 * There is no "state" geographic unit anymore. The parameters now use the term "subdivision1" instead.
 * Other random transformations none of which are particularly interesting.

Folks may wish to view their articles to see if anything got mucked up. They may require refresh's for changes to appear. - <font color="#0A9DC2">~  <font color="#0DC4F2">Ph <font color="#3DD0F5">l <font color="#6EDCF7">o <font color="#9EE8FA">x  18:24, 21 June 2009 (UTC)


 * Template:Age (smw) has crashed in Template:Biography SMW on Abraham Hunsberger (1786-1860) but works on Jacob Kolb Detweiler (1763-1847). Also, for some reason the biography is displaying twice on the page. Template:Age (smw) is also producing some strange results in Template:Biography on pages with info pages. See Samuel Hunsicker (1672-1717), Unknown Klemmer (1672-1717), Melchior Hunsicker (1645-1722), and Elizabeth Hunsicker (c1732-1789). Bill Hunsicker 01:01, 23 June 2009 (UTC)
 * Thanks Bill. Abraham is working now.  I cleaned up Image width but cleaned up too much from that article.  Your Biography SMW thing is in the header for now.  The header is a default thing for newbies and experience folks would just use the contents possibly removing biography if it was redundant to their own hand written bio. As for the info pages, these unfortunately only had partial data in the "birth date" and "death date" fields and Age (SMW) assumes full dates in those two fields.  The info page versions have a single year in the SMW date fields.  It is supposed to be YYYY/MM/DD or nothing at all.  The thing that makes info pages (sort of) work with SMW needs more sophistication to generate more proper dates for age (smw) to work properly but that was not the goal of that scaffolding code.  Anyway, the bot driven process would perform this conversion properly.  - <font color="#0A9DC2">~  <font color="#0DC4F2">Ph <font color="#3DD0F5">l <font color="#6EDCF7">o <font color="#9EE8FA">x   05:12, 23 June 2009 (UTC)

Ancestry autocat
There is now a SMW-based Ancestry from Byzantium test that autocats same. It's part of info categories, and needs to be initialised at emigration. Easy to copy to other ancestries. I used "autocat" rather than "autoprop" because one can have multiple ancestries.

Would be cool if someone figures out how to replace the boring "Ancestry from Ireland" with the Irish flag. rtol 13:30, 22 June 2009 (UTC)


 * Nice idea in principle but inappropriate replacement unless you use a pre-1921 flag, because this wiki uses "Ireland" to mean the whole island. — Robin Patterson (Talk) 10:22, 24 June 2009 (UTC)


 * A [[Image:SmallShamrock.png]] it is then rtol 17:17, 24 June 2009 (UTC)

"Edit this page" vs. "Edit with form"
I checked out the SMW feature some time ago with my great-great-great grandfather, Thomas A. VanBuskirk (1848-1915). I entered data on the form, looked at how it was displayed on the infobox, but I noticed there was no link to edit the form, like on George Spencer Geer (1836)'s article. How is this displayed? And in the future, what are we going to do about the "Edit this page" links? - AMK152 (talk • contribs) 15:47, 29 June 2009 (UTC)
 * It is unnecessary to manually copy paste stuff over from info pages. There is some scaffolding code which moves the data into properties.  You can then remove the scaffolding template and subst in some code that will transfer the data to a page.  This is a manual process for experimentation only.  A real bot run would execute a more thorough check on values, since there is a lot of erroneously coded information on the info pages that needs more sophisticated code than what can be done practically in a template.  If you'd like to try the manual method, see Template:Set_facts_from_info/doc.  To answer your question, you need to employ the showfacts family of templates on the page for the Edit with form menu item to appear.  Examples may be found on the Category:Facts articles- person.  The header and footer templates are really for novice users, and would be included in a boilerplate preload page that the form uses for new articles.  More sophisticated authors might like to delete the biographies template in the header and use their own, or rearrange the placement of sources sections that is performed in footer. - 21:04, 29 June 2009 (UTC)


 * The other thing about the "edit with form" option is that I have found it not appearing until after a couple of saves after the templates have been put in. Thurstan 21:52, 3 July 2009 (UTC)

Order of listing person articles
I wonder whether some of our recent work is losing or neglecting the DEFAULTSORT function. Have a look at http://genealogy.wikia.com/wiki/Special:BrowseData/Facts_articles-_person?_single and see all the Catherines and Dorothys listed under surnames. Are they or should they be the exception - or the rule? Is their listing there determined by anything to do with SMW? — Robin Patterson (Talk) 04:49, 3 July 2009 (UTC)
 * Uh. "Be bold?" Why not just fix it? Thurstan and Rtol are not timid about it and have made several improvements.   16:59, 3 July 2009 (UTC)


 * Fairly unhelpful response. I didn't say anything was broke, so why suggest that I fix something? If something is broke, I don't know what it is (as my "wonder" should have indicated). I was asking whether listing people under surnames was or should be the exception or the rule. I asked whether it was anything to do with SMW. That's not saying something needs fixing. — Robin Patterson (Talk) 06:00, February 21, 2010 (UTC)

Subsequent developments indicate that something in SMW is replacing DEFAULTSORT. But I don't know the extent of that replacement or how much room there is for exceptions, such as in the surname categories, where we certainly don't want everybody listed according to surname. See Forum:Should I nuke Defaultsort lines?. — Robin Patterson (Talk) 06:00, February 21, 2010 (UTC)

I found what was probably the cause, a couple of months later. I fixed it to order by "first name" first. Thurstan changed it to use just "PAGENAME" - which is simpler but perhaps unfortunate for pages starting with words such as "Sir" or "Rev" (which exist - see the latest at Rev Warren Fay (1784-1864)). — Robin Patterson (Talk) 03:24, May 31, 2010 (UTC)

Report from the Medical labs
It is not at all obvious what the nature of the progress in the Phlox laboritories has been, but there has been some very good news in the past few days that ought to be reported.
 * 1) A way was developed to eliminate the double flush thing for typical users (I assume that the mass of typical users will use forms). Folks doing automated stuff may have problems due the way #declare works, but maybe if that is the source of the difficulty I will just eliminate use of #declare.
 * 2) There was success with forms component transculsion experiments. This means the localization nut has been cracked, and I also have a way to share code between detailed forms, an EZ form, and the kitchen sink form. (The current Form:Person represents the kitchen sink- to be replaced with an ultra simplified EZ form.  Folks that want to enter more esoteric data will be invited to open more detailed forms or to go to the kitchen sink version.)
 * 3) A second optimization point is that we now have a way of cutting RTE edit load time down from 45+ seconds to the typical 3-5 seconds that it takes for articles with no fancy templates.

These changes will take quite a few days to fully manifest in code. 21:10, 6 July 2009 (UTC)