Familypedia
Advertisement
Forums: Index > Watercooler > Concepts


Organization[]

I was introduced to the idea of SMW Concepts by Robin a few days ago and have read all the documentation on this site and SMW's site, too. I'm afraid I am missing something. I understand why we would want to do this. I just don't understand the how. I tried to create a user sandbox to play around with it, but is gave me an error message about it being unable to run unless it's in a "Concept:" page. So I started Concept:Sandbox and tried again. It's still not running my queries and I just don't understand why not.

Is there a place where one can go to get a step-by-step tutorial on this?

If not, can someone please give me a nudge in the right direction? Lanica 16:45, June 7, 2010 (UTC)

LATER: I realized that it's not running in 'preview' but it works when I save the page each time. *Thumps head*

How are we planning to use this? Are we trying to replace every category? Is there a framework for page creation being set up or can I just make whatever pages I want to?

For Example: If I enter 'Birth nation::Germany' 'Death nation::United States', would I call that page 'Concept:German Emigration to the United States'. Or 'Concept:United States immigrants from Germany'. It would include all years and could be broken down by dates once it becomes too large. Do we see a need for that? I think this (concepts as a whole) could become a huge project, I don't know where to begin...is there a coordinator? Am I getting way ahead of the plan? Lanica 17:37, June 7, 2010 (UTC)

Thurstan and rtol (and probably Lanica!) are more familiar with the "concept" than I am, but anyone can join in. Lanica can be interim coordinator until we have an election. I guess she has looked at Help:Concepts and at Thurstan's Category:Born in New South Wales example(s). As I think I said on Familypedia:Concepts very recently, it will be highly desirable to have concept pagenames matching category pagenames where the subject-matter is near enough to the same, e.g. Concept:Migrants from Germany to the United States, which I have just created, by moving Lanica's sandbox, and categorized and annotated. I'm working on a related sort of page at Resided_in_New_South_Wales. After we have created, linked, and categorized a few dozen, we will see how to fit them together and what directions we might turn in. — Robin Patterson (Talk) 06:56, June 8, 2010 (UTC)

Ordering names[]

I just added a few pages like Concept:Migrants from the United Kingdom to Australia and the names are all in order by last name, but the index letters are the first. Anyone know how to fix this quirk? Lanica 15:32, June 9, 2010 (UTC)

Excellent use of the concept.
Concepts gather pages with particular properties. "birth nation" is a property of the page rather than the person, so the listing is alphabetic on page name.
If you just want to display those born in the UK and died in Australia, use

{{#ask: [[Birth nation::United Kingdom]] [[Death nation::Australia]]|sort=surname}}

Esther Abrahams (c1767-1846), Mary Ann Adams (c1807-1875), George Edwin Adams (1823-1897), James Ainslie (1808-1860), Georgina Gordon Ainslie (c1845-1929), Amily Annie Ainslie (c1831-1915), Mary Stewart Ainslie (c1830-1925), Mary Ann Albury (1821-1906), John Albury (1823-1855), John Albury (1794-1868), Margaret Ann Albury (1831-1913), Mercy Albury (1834-1884), Charlotte Albury (1819-1891), Frederick Alcock (1807-1869), Jonas Harold Alcock (1843-1933), Richard Alcorn (c1799-1879), John Alderton (1806-1893), William Aldridge (1772-1832), Janet Algie (c1851-1940), Andrew Allan (1798-1837), Mary Amanet (1788-1827), William Amos (1812-1895), Barbara Anderson (1820-1883), William Andrews (c1844-1928), Thomas Ainsworth Anlezark (1771-1834), George Anstey (c1838-1869), William Apps (1835-1911), Eleanor Apps (1815-1854), William Apps (1822-1870), Moses Baker Apps (1833-1875), William Apps (c1812-1886), William Apps (1813-1901), Reuben Apps (1837-1898), Edward Arbon (1819-1896), Thomas Arbon (1823-1872), Frederick Arbon (1846-1910), Phoebe Janet Arbuthnot (1881-1973), William Ernest Archer (1754-1833), Thomas Archer (1790-1850), Ann Armitage (1831-1897), Esther Arndell (1781-1846), Thomas Arndell (1753-1821), John Arndell (1772-1805), Frederick Arnold (c1851-1925), Charlotte Sarah Arnold (c1873-1952), Charles Artlett (1838-1906), Thomas Arundell (1806-1860), Ann Jackson Arundell (1831-1919), Edward Edwin Ashdown (c1836-1904), William Ashenden (1829-1867)... further results

or

{{#ask: [[Birth nation::United Kingdom]] [[Death nation::Australia]]?children-g1|sort=surname}}

 Children-g1
Esther Abrahams (c1767-1846)
Mary Ann Adams (c1807-1875)
George Edwin Adams (1823-1897)
James Ainslie (1808-1860)
Georgina Gordon Ainslie (c1845-1929)
Amily Annie Ainslie (c1831-1915)
Mary Stewart Ainslie (c1830-1925)
Mary Ann Albury (1821-1906)
John Albury (1823-1855)
John Albury (1794-1868)
Margaret Ann Albury (1831-1913)
Mercy Albury (1834-1884)
Charlotte Albury (1819-1891)
Frederick Alcock (1807-1869)
Jonas Harold Alcock (1843-1933)
Richard Alcorn (c1799-1879)
John Alderton (1806-1893)
William Aldridge (1772-1832)
Janet Algie (c1851-1940)
Andrew Allan (1798-1837)
Mary Amanet (1788-1827)
William Amos (1812-1895)
Barbara Anderson (1820-1883)
William Andrews (c1844-1928)
Thomas Ainsworth Anlezark (1771-1834)
George Anstey (c1838-1869)
William Apps (1835-1911)
Eleanor Apps (1815-1854)
William Apps (1822-1870)
Moses Baker Apps (1833-1875)
William Apps (c1812-1886)
William Apps (1813-1901)
Reuben Apps (1837-1898)
Edward Arbon (1819-1896)
Thomas Arbon (1823-1872)
Frederick Arbon (1846-1910)
Phoebe Janet Arbuthnot (1881-1973)
William Ernest Archer (1754-1833)
Thomas Archer (1790-1850)
Ann Armitage (1831-1897)
Esther Arndell (1781-1846)
Thomas Arndell (1753-1821)
John Arndell (1772-1805)
Frederick Arnold (c1851-1925)
Charlotte Sarah Arnold (c1873-1952)
Charles Artlett (1838-1906)
Thomas Arundell (1806-1860)
Ann Jackson Arundell (1831-1919)
Edward Edwin Ashdown (c1836-1904)
William Ashenden (1829-1867)
... further results
rtol 18:12, June 9, 2010 (UTC)
Thank you, Richard. Your expertise in such matters is greatly valued. — Robin Patterson (Talk) 13:21, June 14, 2010 (UTC)
Yes, thanks from me as well. Makes perfect sense...now. ;) Lanica 13:30, June 14, 2010 (UTC)

Combining with category search[]

Many thousands of current articles have been manually or programmatically entered into categories related to places. Not all of them will be caught by the above birth/death coding. We want a way to combine them. The page for displaying the result, as far as I can see, is the article with the same pagename, such as Migrants from the United Kingdom to Australia. I think we can combine a category search with a facts search, and by using magic words and other templates we can get easily-remembered standard templates to apply to any such category with very little manual work, in the same way as we do with the county categories such as {{B-us}}.

So, how to combine in an article the facts search (as above) - {{#ask: [[Birth nation::United Kingdom]] [[Death nation::Australia]]?children-g1|sort=surname}} - and a category search such as:

{{#ask: 
|[[Category:{{BASEPAGENAME}}]] <!-- being, e.g., Migrants from the United Kingdom to Australia -->
| ?Birth date
| ?Birth locality
| ?Death date
| ?Death locality
| ?surname
| ?Father
| ?Mother
| ?joined with-g1
| ?joined with-g2
| ?joined with-g3
| limit=15
}}
[[Category:{{BASEPAGENAME}}| ]]

(Examples of the latter at Resided_in_New_South_Wales and Born in Lanarkshire, both of which I hope to improve by using this combination).

I presume that it needs an "OR" between the category search and the "Birth..." [implied "AND"] "Death...". But do they hang together like that?

I guess we will have to sacrifice the original "sort by surname" if we want sortable columns by various things, but we can make "surname" a separate sortable column as above.

When the result spits out hundreds of pages, it will be time to have a note on the page telling users how they can copy the code to a new page and substitute UK and Australia with smaller areas, for example, to get a more manageable focused list.

Robin Patterson (Talk) 13:21, June 14, 2010 (UTC)

{{#ask: [[Birth nation:Mars]] OR [[Category:Born on Mars]]}} should work.
It may be better, though, to run a bot to change categories to properties. rtol 15:30, June 15, 2010 (UTC)
Bots are not supposed to be able to change a category to something in another namespace, and I wouldn't want them to (for reasons that are partly explained in the first two sentences of this section). Creating a concept page with the same pagename as a category (in bulk), however, would be a good move in principle; but it would be best to give such concept pages workable content right from the start, so it needs planning and discussion. See Category:Born in 1914. — Robin Patterson (Talk) 14:58, June 16, 2010 (UTC)

Selection of columns[]

Why are we including all the additional columns? If the title is 'migrants from' why are we including things that are outside that parameter? I vote for keeping the page as clean as possible: page name and surname (for sorting). Lanica 13:27, June 14, 2010 (UTC)
The additional columns will help searchers who have non-standard information, e.g. don't know the surname but do know the spouse name and death locality (or a few probable localities). If all that you want shown is the page names (in surname order), we can do better with just the category, because you get three columns and therefore need less scrolling. Surname order can be arranged for groups of categories related to person pages, and we can put such pages into those categories using SMW, in the same way as we have done it for surname categories. Then for a big group of people (when we have millions in the database), such as migrants from Europe to Minnesota, we can have a concept that specifies the surname or variants, e.g. Concept:Migrants from Europe to Minnesota named Olson or Ohlson or Olsen. — Robin Patterson (Talk) 14:58, June 16, 2010 (UTC)

Draft proposal to create thousands of concepts[]

Can a bot do this?[]

(Starting with a basic one not combining two or more facts; even one like this for every single fact would take us over 7 million pages, and combining facts would result in billions or bigger numbers)

START
  1. Find a page [[Category:Born in Xyz]] that has not been found previously in this bot run - but if no more of them, STOP
  2. Create page [[Concept:Born in Xyz]] - but if it already exists go back to START
  3. Put the page in [[Category:Concepts about birthplace|Xyz]] and [[Category:Xyz|*Concepts]]
  4. Add to the text {{seealso|Category:Born in Xyz}}
  5. Add the query blah blah SMW SMW blah blah = Xyz etc (this will need to include coding for locality, county, nation, etc - unless we could more simply concentrate on one at a time using categories to find them)
  6. Publish page and return to START

-- Robin Patterson (Talk) 15:05, January 2, 2012 (UTC). Small amendments -- Robin Patterson (Talk) 12:43, January 3, 2012 (UTC)

If not, can it do something similar?[]

Details


Improvements on the above?[]

(It's now 4am and I got out of bed to record this brainwave, so I haven't checked anything much yet.) -- Robin Patterson (Talk) 15:05, January 2, 2012 (UTC)

Other comments?[]

-- Robin Patterson (Talk) 15:05, January 2, 2012 (UTC)

Caution[]

As mentioned above, this could get out of control.

Special:SemanticStatistics told me a few minutes ago that "This wiki contains 7,045,215 property values for a total of 574 different properties". Most of those "property values" (commonly called "facts") are entered by contributors, but some (e.g. age of mother at death) are derived. Each of those facts could have its own concept: Born in 1760, Born in 1761, Born in Ohio, Born in the United States, Female, Married on the 7th day of a month, Buried in November, Birth sourced from IGI, Died at sea, Child of Willem Overmars (c1796-1849), Mother of Hendrikus Overmars (c1837-1893), Middle names "John Henry", Father's given name Walter, Died of consumption, and so on. We already have thousands of categories for many of those facts. Where we have a category, it may be a good idea to create a matching concept (linking to and from it, so as to improve coverage of the database while not all articles fully comply with SMW). Hence my above-suggested initial run. But a fact such as "Child of Willem Overmars (c1796-1849)" is likely to be completely covered on a single existing page (Willem's own page in that example), so no category or concept is worth while.

The next step up is combining two or more facts into one category or concept. We've done a few already, such as the migration-from-one-place-to-another categories and the "Surname in place" categories. Others have been discussed, such as "Born in Ohio in 1860". Combining just two would theoretically produce nearly 25 trillion (half of seven million times seven million) - but would be much less because many combinations are impossible (e.g Born in 1600 and either Born in 1900 or Died in 1500). Some combinations are possible but redundant (e.g. Born in Ohio and Born in the United States): those could have value in, for example, checking for events that had a locality, county, or state without a country listed. Other possible but unlikely combinations could lead to other checks: "Males with given name Elizabeth", for example.

Combining three or more would take us beyond trillions of theoretically valid concept pages.

Most of the resulting concepts would be of interest to so few people (if any) that our time (and Wikia's servers) would be better used in the writing of clear instructions for people to create their own SMW inline queries.

WorldConnect-search.jpg

A search of a database such as WorldConnect somehow produces the sort of combination discussed above. WorldConnect doesn't have an actual page for each of the trillions of combinations that could be entered in a search box. Its software must use some sort of dynamic query system. SMW can do that too, here, now : see Semantic MediaWiki/demo query-subquery and Special:Ask and Help:Displaying information. And we can probably offer searchers more boxes to choose from!

-- Robin Patterson (Talk) 12:47, January 7, 2012 (UTC)

Progress by October 2013[]

Thurstan and I have been trading opinions since I re-examined Lanica's "Migrants from the United Kingdom to Australia" and created similar pages for other target countries, states, and provinces. Thurstan added an improvement (ortiginally to the "UK to France" page) to make more use of the actual concept in addition to a mere table. Then I reread some of the SMW website help pages. Have we used #concept (rather than #ask) - as on http://semantic-mediawiki.org/wiki/Help:Concepts - anywhere? -- Robin Patterson (Talk) 04:57, October 2, 2013 (UTC)

Use for surname variants and categories[]

We make hyphenated and prefixed surname categories into subcategories of the base names. Example at Category:Piper-Smith (surname). We don't yet (but we could) include other sorts of variant surnames. To save people from having to scour subcategories, presumably a concept or simple "ask" could collect everyone. Like this: Smith (surname)/variants?

In fact, couldn't our surname categories all have a subpage like that, whether or not they have variants? Even semi-automatically created, or at least with a standard opening paragraph giving instructions on how to "roll-your-own"??

I'm open to having someone point out the advantage (if any) of using concepts there instead of queries.

-- Robin Patterson (Talk) 01:43, November 8, 2013 (UTC)

Advertisement