Merge pull request #4 from Jaculabilis/develop

Improved package and object model

.gitignore

@@ -102,3 +102,9 @@ ENV/
 
 # Ignore directories in lexicon/
 lexicon/*/
+
+# Ignore vscode
+.vscode
+
+# Ignore a scratch directory
+scratch/

LEXIPYTHON.md

@@ -8,19 +8,64 @@ To play Lexicon, all you really need is something for the scholars to write thei
 
 To aid in playing Lexicon, Lexipython **does** provide the following:
 * Specialized markdown parsing into formatted Lexicon entries.
+* An editor with a live preview of the parsed result and a download button.
 * HTML page generation and interlinking.
-* Handy help pages for rules, session information, statistics, and more.
+* Handy help pages for rules, markdown formatting, session information, and statistics.
 
 Lexipython **does not** provide:
-* Web hosting for the Lexicon pages. The Editor should have a plan for distributing new editions of the Lexicon.
+* Programmatic article submission. The current version of Lexipython does not involve a persistent server process. Players must send their articles to the Editor themselves.
+* Web hosting for the Lexicon pages. The Editor should have a plan for distributing new editions of the Lexicon, such as GitHub Pages.
 * Checks for factual consistency between submitted articles. The Editor is responsible for ensuring scholarly rigor.
 
 ## Using Lexipython
 
-To run a game of Lexicon with Lexipython, clone this repository out to a new folder:
+To run a game of Lexicon with Lexipython, use [git](https://git-scm.com/) to clone this repository out to a new folder.
 ```
-$ git clone https://github.com/Jaculabilis/Lexipython.git [name]
+$ git clone https://github.com/Jaculabilis/Lexipython.git
 ```
 
-Steps for setup:
+Lexipython requires [Python 3](https://www.python.org/downloads/). It will run with only the Python 3 standard library installed, but pagerank statistics will be unavailable without `networkx` installed.
-1. [WIP]
+```
+$ pip install --user networkx
+```
+
+When you have the necessary software installed, open a terminal in the Lexipython directory. You can view the usage of the program with
+```
+$ python lexipython -h
+usage: lexipython [-h] [name] [command]
+
+Lexipython is a Python application for playing the Lexicon RPG.
+
+positional arguments:
+  name        The name of the Lexicon to operate on
+  command     The operation to perform on the Lexicon
+
+optional arguments:
+  -h, --help  show this help message and exit
+
+Run lexipython.py without arguments to list the extant Lexicons.
+
+Available commands:
+
+    init        Create a Lexicon with the provided name
+    build       Build the Lexicon, then exit
+    run         Launch a persistent server managing the Lexicon
+```
+
+Your lexicons are stored in the `lexicon/` folder. Run `python lexipython` to see the status of all lexicons. Except I haven't implemented that yet. Ignore that bit. If you run `python lexipython [name]`, you'll get the status of the named lexicon. That also hasn't been implemented. Whoops!
+
+To create a lexicon, run `python lexipython [name] init` with the name of the lexicon. A folder will be created in `lexicon/` with the given name and some default files will be copied in. You'll need to add a logo image to the folder and edit the config. As players submit articles, place the .txt files in `lexicon/[name]/src/`.
+
+When you finish your initial edits to the config and whenever you want to update the generated HTML files, run `python lexipython [name] build`. Lexipython will regenerate the article pages under `lexicon/[name]/article/` as well as the contents, formatting, rules, session, and statistics pages, and the editor.
+
+To publish the pages, simply copy the lexicon's folder to wherever you're hosting the static files. If you wish, you can leave out the `src/` directory and the status and cfg files. They're are not navigable from the public-facing pages.
+
+The `run` command isn't implemented yet either, and to be honest that probably isn't how you're supposed to implement it in the first place. Ignore it for now.
+
+## Configuring a lexicon
+
+[`lexicon.cfg`](lexipython/resources/lexicon.cfg) contains comments explaining the various config options. `PROMPT` and `SESSION_PAGE` should be written as raw HTML, and will be inserted directly into the page. If you wish to use the Addendums rule explained in the main readme, set `ALLOW_ADDENDA` to `True`. If `SEARCHABLE_FILE` is defined, then the Session page will link to a file with all the articles on one page.
+
+## Other notes
+
+At the end of the build, Lexipython will check for players citing themselves. The program does not fault on these checks, because players may be writing articles as Ersatz Scrivener, or otherwise allowed to cite themselves. Watch out for any unexpected output here.

README.md

@@ -8,11 +8,11 @@ In Lexicon, each player takes on the role of a scholar. You are cranky, opiniona
 
 ## Basic Rules: What Everyone Should Know
 
-1. Each Lexicon has a _topic statement_ that sets the tone for the game. It provides a starting point for shaping the developing world of the Lexicon. As it is a starting point, don't feel contrained to write only about the topics mentioned directly in it.
+1. Each Lexicon has a **topic statement** that sets the tone for the game. It provides a starting point for shaping the developing world of the Lexicon. As it is a starting point, don't feel contrained to write only about the topics mentioned directly in it.
 
-1. Articles are sorted under an _index_, a grouping of letters. An article is in an index if its first letter is in that group of letters. "The", "A", and "An" aren't counted in indexing.  _Example: One of the indices is JKL. An article titled 'The Jabberwock' would index under JKL, not T's index._
+1. Articles are sorted under an **index**, a grouping of letters. An article is in an index if its first letter is in that group of letters. "The", "A", and "An" aren't counted in indexing.  _Example: Two indices are JKL and TUV. An article titled 'The Jabberwock' would index under JKL, not TUV._
 
-    1. Until the game is over, some of the articles will have been cited, but not yet written. These are called _phantom_ articles. A phantom article has a title, which is defined by the first citation to it, but no content.
+    1. Until the game is over, some of the articles will have been cited, but not yet written. These are called **phantom articles**. A phantom article has a title, which is defined by the first citation to it, but no content.
 
     1. Generally, an index has a number of "slots" equal to the number of players. When an article is first written or cited, it takes up one slot in its corresponding index.
 
@@ -26,7 +26,9 @@ In Lexicon, each player takes on the role of a scholar. You are cranky, opiniona
 
     1. There are no hard and fast rules about length, but it is recommended that the Editor enforce a maximum word limit. In general, aiming for 200-300 words is ideal.
 
-    1. You must respect and not contradict the factual content of all written articles. You may introduce new facts that put things in a new light, provide alternative interpretations, or flesh out unexplained details in unexpected ways; but you must not _contradict_ what has been previously established as fact. Use the "yes, and" rule from improv acting: accept what your fellow scholars have written and add to it in new ways, rather than trying to undo their work. This rule includes facts that have been established in written articles about the topics of phantom articles.
+    1. You must respect and not contradict the factual content of all written articles. You may introduce new facts that put things in a new light, provide alternative interpretations, or flesh out unexplained details in unexpected ways; but you must not _contradict_ what has been previously established as fact. Use the "yes, and" rule from improv acting: accept what your fellow scholars have written and add to it in new ways, rather than trying to undo their work.
+
+        1. This rule includes facts that have been established in other, written articles about the topics of phantom articles. When you set out to write a phantom article, be sure to check what's been said about the topic already. Lexipython will list the articles that have cited your article.
 
 1. Each article will cite other articles in the Lexicon.
 
@@ -34,16 +36,18 @@ In Lexicon, each player takes on the role of a scholar. You are cranky, opiniona
 
     1. As a corollary, you may not write phantom articles that you have cited. If you cite an article and then write it later, your former article now cites you, which is forbidden per the above.
 
-    1. On the first turn, there are no written articles. Your first article must cite _exactly two_ phantom articles.
+    1. On the first turn, there are no written articles. Your first article must cite **exactly two** phantom articles.
 
-    1. On subsequent turns, your article must cite _exactly two_ phantoms, but you can cite phantoms that already exist. Your article must also cite _at least one_ written article. You can cite more than one.
+    1. On subsequent turns, your article must cite **exactly two** phantoms, but you can cite phantoms that already exist. Your article must also cite **at least one** written article. You can cite more than one.
 
-    1. On the penultimate turn, you must cite _exactly one_ phantom article and _at least two_ written articles.
+    1. On the penultimate turn, you must cite **exactly one** phantom article and **at least two** written articles.
 
-    1. On the final turn, you must cite _at least three_ written articles.
+    1. On the final turn, you must cite **at least three** written articles.
 
 1. As the game goes on, it may come to pass that a player must write an article in an index, but that index is full, and that player has already cited all the phantoms in it. When this happens, the player instead writes their article as **Ersatz Scrivener**, radical skeptic. Ersatz does not believe in the existence of whatever he is writing about, no matter how obvious it seems to others or how central it is in the developing history of the world. For Ersatz, all references, testimony, etc. with regard to its existence are tragic delusion at best or malicious lies at worst. Unlike the other scholars, Ersatz does not treat the research of his peers as fact, because he does not believe he has peers. Players writing articles as Ersatz are encouraged to lambast the amateur work of his misguided "collaborators".
 
+1. Finally, the rules are always subject to the discretion of the Editor.
+
 ## Procedural Rules: Running the Game
 
 ### The Editor
@@ -60,7 +64,7 @@ The player running the game is the Editor. The Editor should handle the followin
 
 * **Topic statement.** The topic statement should be vague, but give the players some hooks to begin writing. Examples: "You are all revisionist scholars from the Paleotechnic Era arguing about how the Void Ghost Rebellion led to the overthrow of the cyber-gnostic theocracy and the establishment of the Third Republic"; "In the wake of the Quartile Reformation, you are scholars investigating the influence of Remigrationism on the Disquietists". What happened to the first two Republics or what Remigrationism is are left open for the players to determine.
 
-* **Indices and turns.** In general, the Editor will decide on a number of turns and divide the alphabet into that many indices. Each player then takes one turn in each index. A game of 6 or 8 turns is suggested. _Example: An 8-turn game over the indices ABC/DEF/GHI/JKL/MNO/PQRS/TUV/QXYZ._ The Editor should determine how much time the players can devote to playing Lexicon and set a time limit on turns accordingly.
+* **Indices and turns.** In general, the Editor will decide on a number of turns and divide the alphabet into that many indices. Each player then takes one turn in each index. A game of 6 or 8 turns is suggested. _Example: An 8-turn game over the indices ABC/DEF/GHI/JKL/MNO/PQRS/TUV/WXYZ._ The Editor should determine how much time the players can devote to playing Lexicon and set a time limit on turns accordingly.
 
 * **Index assignments.** Each turn, the Editor should assign each player to an index. Unless players have a method of coordinating who is writing what article, it is suggested that the Editor always assign players to write in different indices. The easiest way to do this is to distribute players randomly across the indices for the first turn, then move them through the indices in order, wrapping around to the top from the bottom.
 
@@ -74,7 +78,7 @@ How the game develops is entirely up to the players, and your group may have a d
 
 * Even if articles don't get too long, having too many articles on one subject can lead to the same problem of writing on the topic becoming too hard to do consistently. Avoid having multiple articles about the same thing, and avoid having too many articles about different facets of one particular element of the world.
 
-* Encyclopedias are written about things in the past. Players may, of course, want to mention how something in the past still affects the world in the present day. However, if players begin to write about purely contemporary things or events, the Lexicon shifts from an _encyclopedic_ work to a _narrative_ one. If that's what you want out of the game, go ahead and do so, but writing about an ongoing narrative insead of settled history introduce the additional complication of keeping abreast of the current state of the plot. It is more difficult for players to avoid contradiction when the facts are changing as they write.
+* Encyclopedias are written about things in the past. Players may, of course, want to mention how something in the past still affects the world in the present day. However, if players begin to write about purely contemporary things or events, the Lexicon shifts from an _encyclopedic_ work to a _narrative_ one. If that's what you want out of the game, go ahead and do so, but writing about an ongoing narrative insead of settled history introduces the additional complication of keeping abreast of the current state of the plot. It is more difficult for players to avoid contradiction when the facts are changing as they write.
 
 * Articles whose titles do not begin with a character in any index pattern are sorted to the "&c" index. This usually includes numbers and symbols. If the Editor wants to make purposive use of this, they can assign players to it as an index.
 
@@ -82,7 +86,9 @@ How the game develops is entirely up to the players, and your group may have a d
 
 The Editor is always free to alter the game procedures when it would make for a better game. The following are some known rule variations:
 
-* **Follow the Phantoms:** Players make two phantom citations on the first turn. On subsequent turns, rather than choosing from phantoms and open slots in an assigned index, players must write an existing phantom. Until all slots are full, players must make one of their phantom citations to a new phantom article and one to an existing phantom.
+* **Follow the Phantoms:** Players make phantom citations as normal on the first turn. On subsequent turns, rather than choosing from phantoms and open slots in an assigned index, players must write an existing phantom. Until all slots are full, players must make one of their phantom citations to a new phantom article and one to an existing phantom.
+
+* **Addendums:** In addition to writing new and phantom articles, players can write articles with the same title as an already-written article. The content of these "addendum" articles is added as a postscript at the bottom of the first article written under that title. Addendums can legally cite what their author can cite, not what the main article's author can cite.
 
 * Occasionally, if more players make a citation to an index than there are open slots, the index will be over capacity. If the Editor is assigning players to indices in order, the Editor may need to shift players' index assignments around. This may also be useful for decreasing the number of Ersatz articles, if a player can't write in their assigned index but could write in another.
 

lexipython/__main__.py

@@ -8,9 +8,9 @@ import argparse
 import os
 import re
 import json
-from src.article import LexiconArticle
+from article import LexiconArticle
-from src import build
+import build
-from src import utils
+import utils
 
 def is_lexicon(name):
 	"""
@@ -136,13 +136,13 @@ def command_init(name):
 	# Edit the name field
 	config = re.sub("Lexicon Title", "Lexicon {}".format(name), config)
 	# Create the Lexicon's config file
-	with open(os.path.join(lex_path, "lexicon.cfg"), "w") as config_file:
+	with open(os.path.join(lex_path, "lexicon.cfg"), "w", newline='') as config_file:
 		config_file.write(config)
 	# Copy the CSS file
-	with open(os.path.join(lex_path, "lexicon.css"), "w") as css_file:
+	with open(os.path.join(lex_path, "lexicon.css"), "w", newline='') as css_file:
 		css_file.write(utils.load_resource("lexicon.css"))
 	# Create an example page
-	with open(os.path.join(lex_path, "src", "example-page.txt"), "w") as destfile:
+	with open(os.path.join(lex_path, "src", "example-page.txt"), "w", newline='') as destfile:
 		destfile.write(utils.load_resource("example-page.txt"))
 	# Create an empty status file
 	open(os.path.join(lex_path, "status"), "w").close()

lexipython/article.py

@@ -1,7 +1,7 @@
 import os
 import sys
 import re
-import src.utils as utils
+import utils
 
 class LexiconCitation:
 	"""
@@ -167,11 +167,25 @@ class LexiconArticle:
 		return articles
 
 	@staticmethod
-	def interlink(lexicon_articles):
+	def interlink(lexicon_articles, config):
 		"""
 		Fills out fields on articles that require other articles for context.
 		Creates phantom articles.
 		"""
+		# Preliminary assertion that title/turn is unique
+		keys = set()
+		for article in lexicon_articles:
+			if config['ALLOW_ADDENDA'].lower() == "true":
+				key = (article.title, article.turn)
+				if key in keys:
+					raise ValueError("Found two articles with title '{}' and turn '{}'".format(
+						*key))
+			else:
+				key = article.title
+				if key in keys:
+					raise ValueError("Found two articles with title '{}'".format(
+						article.title))
+			keys.add(key)
 		# Sort out which articles are addendums and which titles are phantoms
 		written_titles = set()
 		cited_titles = set()
@@ -221,8 +235,7 @@ class LexiconArticle:
 		content += "<div class=\"contentblock\"><h1>{}</h1>{}</div>\n".format(
 			self.title, main_body)
 		# Build the main citation content block
-		main_citations = self.build_default_citeblock(
+		main_citations = self.build_default_citeblock()
-			self.prev_article, self.next_article)
 		if main_citations:
 			content += "<div class=\"contentblock citeblock\">{}</div>\n".format(
 				main_citations)
@@ -230,10 +243,16 @@ class LexiconArticle:
 		for addendum in self.addendums:
 			add_body = addendum.build_default_article_body()
 			content += "<div class=\"contentblock\">{}</div>\n".format(add_body)
-			add_citations = addendum.build_default_citeblock(None, None)
+			add_citations = addendum.build_default_citeblock()
 			if add_citations:
 				content += "<div class=\"contentblock\">{}</div>\n".format(
 					add_citations)
+		# Build the prev/next block
+		prev_next = self.build_prev_next_block(
+			self.prev_article, self.next_article)
+		if prev_next:
+			content += "<div class=\"contentblock citeblock\">{}</div>\n".format(
+				prev_next)
 		return content
 
 	def build_default_article_body(self):
@@ -247,23 +266,12 @@ class LexiconArticle:
 		}
 		return self.content.format(**format_map)
 
-	def build_default_citeblock(self, prev_article, next_article):
+	def build_default_citeblock(self):
 		"""
-		Builds the contents of a citation contentblock. For each defined target,
+		Builds the contents of a citation contentblock. Skips sections with no
-		links the target page as Previous or Next. Prev/next and cites/citedby
+		content.
-		elements are not included if they have no content.
 		"""
 		content = ""
-		# Prev/next links:
-		if next_article is not None or prev_article is not None:
-			prev_link = ("<a {0.link_class} href=\"{0.title_filesafe}.html\">&#8592; Previous</a>".format(
-				prev_article)
-				if prev_article is not None else "")
-			next_link = ("<a {0.link_class} href=\"{0.title_filesafe}.html\">Next &#8594;</a>".format(
-				next_article)
-				if next_article is not None else "")
-			content += "<table><tr>\n<td>{}</td>\n<td>{}</td>\n</table></tr>\n".format(
-				prev_link, next_link)
 		# Citations
 		cites_titles = set()
 		cites_links = []
@@ -289,3 +297,20 @@ class LexiconArticle:
 			content += "<p>Cited by: {}</p>\n".format(citedby_str)
 
 		return content
+
+	def build_prev_next_block(self, prev_article, next_article):
+		"""
+		For each defined target, links the target page as Previous or Next.
+		"""
+		content = ""
+		# Prev/next links:
+		if next_article is not None or prev_article is not None:
+			prev_link = ("<a {0.link_class} href=\"{0.title_filesafe}.html\">&#8592; Previous</a>".format(
+				prev_article)
+				if prev_article is not None else "")
+			next_link = ("<a {0.link_class} href=\"{0.title_filesafe}.html\">Next &#8594;</a>".format(
+				next_article)
+				if next_article is not None else "")
+			content += "<table><tr>\n<td>{}</td>\n<td>{}</td>\n</tr></table>\n".format(
+				prev_link, next_link)
+		return content

lexipython/build.py

@@ -1,12 +1,12 @@
-import sys		# For argv and stderr
+# Standard library imports
 import os		# For reading directories
 import re		# For parsing lex content
-import io		# For writing pages out as UTF-8
-import networkx # For pagerank analytics
-from collections import defaultdict # For rank inversion in statistics
 
-from src import utils
+# Application imports
-from src.article import LexiconArticle
+import utils
+from article import LexiconArticle
+from statistics import LexiconStatistics
+
 
 class LexiconPage:
 	"""
@@ -28,11 +28,21 @@ class LexiconPage:
 		total_kwargs = {**self.kwargs, **kwargs}
 		return self.skeleton.format(**total_kwargs)
 
-def build_contents_page(page, articles, index_list):
+def article_matches_index(index_type, pattern, article):
+	if index_type == "char":
+		return utils.titlesort(article.title)[0].upper() in pattern.upper()
+	if index_type == "prefix":
+		return article.title.startswith(pattern)
+	if index_type == "etc":
+		return True
+	raise ValueError("Unknown index type: '{}'".format(index_type))
+
+def build_contents_page(config, page, articles):
 	"""
 	Builds the full HTML of the contents page.
 	"""
 	content = "<div class=\"contentblock\">"
+
 	# Head the contents page with counts of written and phantom articles
 	phantom_count = len([article for article in articles if article.player is None])
 	if phantom_count == 0:
@@ -40,32 +50,68 @@ def build_contents_page(page, articles, index_list):
 	else:
 		content += "<p>There are <b>{0}</b> entries, <b>{1}</b> written and <b>{2}</b> phantom.</p>\n".format(
 			len(articles), len(articles) - phantom_count, phantom_count)
+
 	# Prepare article links
 	link_by_title = {article.title : "<a href=\"../article/{1}.html\"{2}>{0}</a>".format(
 			article.title, article.title_filesafe,
 			" class=\"phantom\"" if article.player is None else "")
 			for article in articles}
-	# Write the articles in alphabetical order
+
-	content += utils.load_resource("contents.html")
+	# Determine index order
-	content += "<div id=\"index-order\" style=\"display:none\">\n<ul>\n"
+	indices = config['INDEX_LIST'].split("\n")
-	indices = index_list.split("\n")
+	index_by_pri = {}
-	alphabetical_order = sorted(
+	index_list_order = []
+	for index in indices:
+		match = re.match(r"([^[:]+)(\[([-\d]+)\])?:(.+)", index)
+		index_type = match.group(1)
+		pattern = match.group(4)
+		try:
+			pri_s = match.group(3)
+			pri = int(pri_s) if pri_s else 0
+		except:
+			raise TypeError("Could not parse index pri '{}' in '{}'".format(pri_s, index))
+		if pri not in index_by_pri:
+			index_by_pri[pri] = []
+		index_by_pri[pri].append((index_type, pattern))
+		index_list_order.append(pattern)
+
+	# Assign articles to indices
+	articles_by_index = {pattern: [] for pattern in index_list_order}
+	titlesort_order = sorted(
 		articles,
 		key=lambda a: utils.titlesort(a.title))
-	check_off = list(alphabetical_order)
+	for article in titlesort_order:
-	for index_str in indices:
+		# Find the first index that matches
-		content += "<h3>{0}</h3>\n".format(index_str)
+		matched = False
-		for article in alphabetical_order:
+		for pri, indices in sorted(index_by_pri.items(), reverse=True):
-			if (utils.titlesort(article.title)[0].upper() in index_str):
+			for index_type, pattern in indices:
-				check_off.remove(article)
+				# Try to match the index
-				content += "<li>{}</li>\n".format(link_by_title[article.title])
+				if article_matches_index(index_type, pattern, article):
-	if len(check_off) > 0:
+					articles_by_index[pattern].append(article)
-		content += "<h3>&c.</h3>\n"
+					matched = True
-		for article in check_off:
+				# Break out once a match is found
+				if matched:
+					break
+			if matched:
+				break
+		if not matched:
+			raise KeyError("No index matched article '{}'".format(article.title))
+
+	# Write index order div
+	content += utils.load_resource("contents.html")
+	content += "<div id=\"index-order\" style=\"display:{}\">\n<ul>\n".format(
+		"block" if config["DEFAULT_SORT"] == "index" else "none")
+	for pattern in index_list_order:
+		# Write the index header
+		content += "<h3>{0}</h3>\n".format(pattern)
+		# Write all matches articles
+		for article in articles_by_index[pattern]:
 			content += "<li>{}</li>\n".format(link_by_title[article.title])
 	content += "</ul>\n</div>\n"
-	# Write the articles in turn order
+
-	content += "<div id=\"turn-order\" style=\"display:none\">\n<ul>\n"
+	# Write turn order div
+	content += "<div id=\"turn-order\" style=\"display:{}\">\n<ul>\n".format(
+		"block" if config["DEFAULT_SORT"] == "turn" else "none")
 	turn_numbers = [article.turn for article in articles if article.player is not None]
 	first_turn, last_turn = min(turn_numbers), max(turn_numbers)
 	turn_order = sorted(
@@ -83,6 +129,31 @@ def build_contents_page(page, articles, index_list):
 		for article in check_off:
 			content += "<li>{}</li>\n".format(link_by_title[article.title])
 	content += "</ul>\n</div>\n"
+
+	# Write by-player div
+	content += "<div id=\"player-order\" style=\"display:{}\">\n<ul>\n".format(
+		"block" if config["DEFAULT_SORT"] == "player" else "none")
+	articles_by_player = {}
+	extant_phantoms = False
+	for article in turn_order:
+		if article.player is not None:
+			if article.player not in articles_by_player:
+				articles_by_player[article.player] = []
+			articles_by_player[article.player].append(article)
+		else:
+			extant_phantoms = True
+	for player, player_articles in sorted(articles_by_player.items()):
+		content += "<h3>{0}</h3>\n".format(player)
+		for article in player_articles:
+			content += "<li>{}</li>\n".format(link_by_title[article.title])
+	if extant_phantoms:
+		content += "<h3>Unwritten</h3>\n"
+		for article in titlesort_order:
+			if article.player is None:
+				content += "<li>{}</li>\n".format(link_by_title[article.title])
+	content += "</ul>\n</div>\n"
+
+	content += "</div>\n"
 	# Fill in the page skeleton
 	return page.format(title="Index", content=content)
 
@@ -102,199 +173,39 @@ def build_formatting_page(page):
 	# Fill in the entry skeleton
 	return page.format(title="Formatting", content=content)
 
-def build_session_page(page, session_content):
+def build_session_page(page, config):
 	"""
 	Builds the full HTML of the session page.
 	"""
-	# Fill in the entry skeleton
+	# Misc links
-	content = "<div class=\"contentblock\">{}</div>".format(session_content)
+	content =  '<div class="contentblock misclinks"><table><tr>\n'
+	content += '<td><a href="../editor.html">Editor</a></td>\n'
+	if config['SEARCHABLE_FILE']:
+		content += '<td><a href="../{}">Compiled</a></td>\n'.format(config['SEARCHABLE_FILE'].strip())
+	content += '</tr></table></div>\n'
+	# Session content
+	content += "<div class=\"contentblock\">{}</div>".format(config["SESSION_PAGE"])
+
 	return page.format(title="Session", content=content)
 
-def reverse_statistics_dict(stats, reverse=True):
+def build_statistics_page(config, page, articles):
-	"""
+	# Read the config file for which stats to publish.
-	Transforms a dictionary mapping titles to a value into a list of values
+	lines = config['STATISTICS'].split("\n")
-	and lists of titles. The list is sorted by the value, and the titles are
+	stats = []
-	sorted alphabetically.
+	for line in lines:
-	"""
+		stat, toggle = line.split()
-	rev = {}
+		if toggle == "on":
-	for key, value in stats.items():
+			stats.append("stat_" + stat)
-		if value not in rev:
-			rev[value] = []
-		rev[value].append(key)
-	for key, value in rev.items():
-		rev[key] = sorted(value, key=lambda t: utils.titlesort(t))
-	return sorted(rev.items(), key=lambda x:x[0], reverse=reverse)
 
-def itemize(stats_list):
+	# Create all the stats blocks.
-	return map(lambda x: "{0} &ndash; {1}".format(x[0], "; ".join(x[1])), stats_list)
+	lexicon_stats = LexiconStatistics(articles)
-
+	stats_blocks = []
-def build_statistics_page(page, articles):
+	for stat in stats:
-	"""
+		if hasattr(lexicon_stats, stat):
-	Builds the full HTML of the statistics page.
+			stats_blocks.append(getattr(lexicon_stats, stat)())
-
+		else:
-	The existence of addendum articles complicates how some statistics are
+			print("ERROR: Bad stat {}".format(stat))
-	computed. An addendum is an article, with its own author, body, and
+	content = "\n".join(stats_blocks)
-	citations, but in a Lexicon it exists appended to another article. To handle
-	this, we distinguish an _article_ from a _page_. An article is a unit parsed
-	from a single source file. A page is a main article and all addendums under
-	the same title.
-	"""
-	min_turn = 0
-	max_turn = 0
-	article_by_title = {}
-	page_by_title = {}
-	players = set()
-	for main_article in articles:
-		key = main_article.title
-		page_by_title[key] = [main_article]
-		page_by_title[key].extend(main_article.addendums)
-		for article in [main_article] + main_article.addendums:
-			# Disambiguate articles by appending turn number to the title
-			key = "{0.title} (T{0.turn})".format(article)
-			article_by_title[key] = article
-			if article.player is not None:
-				min_turn = min(min_turn, article.turn)
-				max_turn = max(max_turn, article.turn)
-				players.add(article.player)
-	content = ""
-	stat_block = "<div class=\"contentblock\"><u>{0}</u><br>{1}</div>\n"
-
-	# Top pages by pagerank
-	# Compute pagerank for each page, including all articles
-	G = networkx.Graph()
-	for page_title, articles in page_by_title.items():
-		for article in articles:
-			for citation in article.citations:
-				G.add_edge(page_title, citation.target)
-	pagerank_by_title = networkx.pagerank(G)
-	for page_title, articles in page_by_title.items():
-		if page_title not in pagerank_by_title:
-			pagerank_by_title[page_title] = 0
-	# Get the top ten articles by pagerank
-	top_pageranks = reverse_statistics_dict(pagerank_by_title)[:10]
-	# Replace the pageranks with ordinals
-	top_ranked = enumerate(map(lambda x: x[1], top_pageranks), start=1)
-	# Format the ranks into strings
-	top_ranked_items = itemize(top_ranked)
-	# Write the statistics to the page
-	content += stat_block.format(
-		"Top 10 articles by page rank:",
-		"<br>".join(top_ranked_items))
-
-	# Pages cited/cited by
-	pages_cited    = {page_title: set() for page_title in page_by_title.keys()}
-	pages_cited_by = {page_title: set() for page_title in page_by_title.keys()}
-	for page_title, articles in page_by_title.items():
-		for article in articles:
-			for citation in article.citations:
-				pages_cited[page_title].add(citation.target)
-				pages_cited_by[citation.target].add(page_title)
-	for page_title, cite_titles in pages_cited.items():
-		pages_cited[page_title] = len(cite_titles)
-	for page_title, cite_titles in pages_cited_by.items():
-		pages_cited_by[page_title] = len(cite_titles)
-
-	top_citations = reverse_statistics_dict(pages_cited)[:3]
-	top_citations_items = itemize(top_citations)
-	content += stat_block.format(
-		"Cited the most pages:",
-		"<br>".join(top_citations_items))
-	top_cited = reverse_statistics_dict(pages_cited_by)[:3]
-	top_cited_items = itemize(top_cited)
-	content += stat_block.format(
-		"Cited by the most pages:",
-		"<br>".join(top_cited_items))
-
-	# Top article length
-	article_length_by_title = {}
-	cumulative_article_length_by_turn = {
-		turn_num: 0
-		for turn_num in range(min_turn, max_turn + 1)
-	}
-	for article_title, article in article_by_title.items():
-		format_map = {
-			"c"+str(c.id): c.text
-			for c in article.citations
-		}
-		plain_content = article.content.format(**format_map)
-		word_count = len(plain_content.split())
-		article_length_by_title[article_title] = word_count
-		for turn_num in range(min_turn, max_turn + 1):
-			if article.turn <= turn_num:
-				cumulative_article_length_by_turn[turn_num] += word_count
-	top_length = reverse_statistics_dict(article_length_by_title)[:3]
-	top_length_items = itemize(top_length)
-	content += stat_block.format(
-		"Longest articles:",
-		"<br>".join(top_length_items))
-
-	# Total word count
-	len_list = [(str(k), [str(v)]) for k,v in cumulative_article_length_by_turn.items()]
-	content += stat_block.format(
-		"Aggregate word count by turn:",
-		"<br>".join(itemize(len_list)))
-
-	# Player pageranks
-	pagerank_by_player = {player: 0 for player in players}
-	for page_title, articles in page_by_title.items():
-		page_author = articles[0].player
-		if page_author is not None:
-			pagerank_by_player[page_author] += pagerank_by_title[page_title]
-	for player, pagerank in pagerank_by_player.items():
-		pagerank_by_player[player] = round(pagerank, 3)
-	player_rank = reverse_statistics_dict(pagerank_by_player)
-	player_rank_items = itemize(player_rank)
-	content += stat_block.format(
-		"Player aggregate page rank:",
-		"<br>".join(player_rank_items))
-
-	# Player citations made
-	pages_cited_by_player = {player: 0 for player in players}
-	for article_title, article in article_by_title.items():
-		if article.player is not None:
-			pages_cited_by_player[article.player] += len(article.citations)
-	player_cites_made_ranks = reverse_statistics_dict(pages_cited_by_player)
-	player_cites_made_items = itemize(player_cites_made_ranks)
-	content += "<div class=\"contentblock\">\n"
-	content += "<u>Citations made by player:</u><br>\n"
-	content += "<br>\n".join(player_cites_made_items)
-	content += "</div>\n"
-
-	# Player cited count
-	pages_cited_by_by_player = {player: 0 for player in players}
-	for page_title, articles in page_by_title.items():
-		page_author = articles[0].player
-		if page_author is not None:
-			pages_cited_by_by_player[page_author] += len(articles[0].citedby)
-	cited_times_ranked = reverse_statistics_dict(pages_cited_by_by_player)
-	cited_times_items = itemize(cited_times_ranked)
-	content += "<div class=\"contentblock\">\n"
-	content += "<u>Citations made to article by player:</u><br>\n"
-	content += "<br>\n".join(cited_times_items)
-	content += "</div>\n"
-
-	# Lowest pagerank of written articles
-	exclude = [a.title for a in articles if a.player is None]
-	rank_by_written_only = {k:v for k,v in pagerank_by_title.items() if k not in exclude}
-	pageranks = reverse_statistics_dict(rank_by_written_only)
-	bot_ranked = list(enumerate(map(lambda x: x[1], pageranks), start=1))[-10:]
-	# Format the ranks into strings
-	bot_ranked_items = itemize(bot_ranked)
-	content += "<div class=\"contentblock\">\n"
-	content += "<u>Bottom 10 articles by pagerank:</u><br>\n"
-	content += "<br>\n".join(bot_ranked_items)
-	content += "</div>\n"
-
-	# Undercited articles
-	undercited = {
-		page_title: len(articles[0].citedby)
-		for page_title, articles in page_by_title.items()
-		if len(articles[0].citedby) < 2}
-	undercited_items = itemize(reverse_statistics_dict(undercited))
-	content += "<div class=\"contentblock\">\n"
-	content += "<u>Undercited articles:</u><br>\n"
-	content += "<br>\n".join(undercited_items)
-	content += "</div>\n"
 
 	# Fill in the entry skeleton
 	return page.format(title="Statistics", content=content)
@@ -398,8 +309,6 @@ def latex_from_markdown(raw_content):
 		para = re.sub("&", "\\&", para)
 		para = re.sub(r"\"(?=\w)", "``", para)
 		para = re.sub(r"(?<=\w)\"", "''", para)
-		para = re.sub(r"%", "\\%", para)
-		para = re.sub(r"#", "\\#", para)
 		# Replace bold and italic marks with commands
 		para = re.sub(r"//([^/]+)//", r"\\textit{\1}", para)
 		para = re.sub(r"\*\*([^*]+)\*\*", r"\\textbf{\1}", para)
@@ -432,9 +341,9 @@ def latex_from_directory(directory):
 			with open(path, "r", encoding="utf8") as src_file:
 				raw = src_file.read()
 				title, turn, latex = latex_from_markdown(raw)
-				if turn not in articles:
+				if title not in articles:
-					articles[turn] = {}
+					articles[title] = {}
-				articles[turn][title] = latex
+				articles[title][turn] = latex
 
 	# Write the preamble
 	content = "\\documentclass[12pt,a4paper,twocolumn,twoside]{article}\n"\
@@ -442,22 +351,14 @@ def latex_from_directory(directory):
 		"\\begin{document}\n"\
 		"\n"
 
-	for turn in sorted(articles.keys()):
+	for title in sorted(articles.keys(), key=lambda t: utils.titlesort(t)):
-		for title in sorted(articles[turn].keys(), key=lambda t: utils.titlesort(t)):
-			latex = articles[turn][title]
-			content += "\\section*{{{}}}\n".format(title)
-			content += "\\label{{{}}}\n\n".format(hash(title))
-			content += latex
-	"""
-	for article in sorted(articles, key=lambda a: (a.turn, utils.titlesort(a.title))):
-		title = article.title
 		under_title = articles[title]
 		turns = sorted(under_title.keys())
 		latex = under_title[turns[0]]
 
 		# Section header
-		content += "\\section*{{{}}}\n\n".format(title)
 		content += "\\label{{{}}}\n".format(hash(title))
+		content += "\\section*{{{}}}\n\n".format(title)
 
 		# Section content
 		#format_map = {
@@ -478,7 +379,7 @@ def latex_from_directory(directory):
 			#	for c in addendum.citations
 			#}
 			#article_content = addendum.content.format(**format_map)
-			content += latex"""
+			content += latex
 
 	content += "\\end{document}"
 
@@ -487,6 +388,15 @@ def latex_from_directory(directory):
 
 	return content
 
+def parse_sort_type(sort):
+	if sort in "?byindex":
+		return "?byindex"
+	if sort in "?byturn":
+		return "?byturn"
+	if sort in "?byplayer":
+		return "?byplayer"
+	return ""
+
 def build_all(path_prefix, lexicon_name):
 	"""
 	Builds all browsable articles and pages in the Lexicon.
@@ -500,13 +410,13 @@ def build_all(path_prefix, lexicon_name):
 		lexicon=config["LEXICON_TITLE"],
 		logo=config["LOGO_FILENAME"],
 		prompt=config["PROMPT"],
-		sort=config["DEFAULT_SORT"])
+		sort=parse_sort_type(config["DEFAULT_SORT"]))
 	# Parse the written articles
 	articles = LexiconArticle.parse_from_directory(os.path.join(lex_path, "src"))
 	# Once they've been populated, the articles list has the titles of all articles
 	# Sort this by turn before title so prev/next links run in turn order
 	articles = sorted(
-		LexiconArticle.interlink(articles),
+		LexiconArticle.interlink(articles, config),
 		key=lambda a: (a.turn, utils.titlesort(a.title)))
 
 	def pathto(*els):
@@ -514,9 +424,9 @@ def build_all(path_prefix, lexicon_name):
 
 	# Write the redirect page
 	print("Writing redirect page...")
-	with open(pathto("index.html"), "w", encoding="utf8") as f:
+	with open(pathto("index.html"), "w", encoding="utf8", newline='') as f:
 		f.write(utils.load_resource("redirect.html").format(
-			lexicon=config["LEXICON_TITLE"], sort=config["DEFAULT_SORT"]))
+			lexicon=config["LEXICON_TITLE"], sort=parse_sort_type(config["DEFAULT_SORT"])))
 
 	# Write the article pages
 	print("Deleting old article pages...")
@@ -527,7 +437,7 @@ def build_all(path_prefix, lexicon_name):
 	l = len(articles)
 	for idx in range(l):
 		article = articles[idx]
-		with open(pathto("article", article.title_filesafe + ".html"), "w", encoding="utf-8") as f:
+		with open(pathto("article", article.title_filesafe + ".html"), "w", encoding="utf-8", newline='') as f:
 			content = article.build_default_content()
 			article_html = page.format(
 				title = article.title,
@@ -537,29 +447,29 @@ def build_all(path_prefix, lexicon_name):
 
 	# Write default pages
 	print("Writing default pages...")
-	with open(pathto("contents", "index.html"), "w", encoding="utf-8") as f:
+	with open(pathto("contents", "index.html"), "w", encoding="utf-8", newline='') as f:
-		f.write(build_contents_page(page, articles, config["INDEX_LIST"]))
+		f.write(build_contents_page(config, page, articles))
 	print("    Wrote Contents")
-	with open(pathto("rules", "index.html"), "w", encoding="utf-8") as f:
+	with open(pathto("rules", "index.html"), "w", encoding="utf-8", newline='') as f:
 		f.write(build_rules_page(page))
 	print("    Wrote Rules")
-	with open(pathto("formatting", "index.html"), "w", encoding="utf-8") as f:
+	with open(pathto("formatting", "index.html"), "w", encoding="utf-8", newline='') as f:
 		f.write(build_formatting_page(page))
 	print("    Wrote Formatting")
-	with open(pathto("session", "index.html"), "w", encoding="utf-8") as f:
+	with open(pathto("session", "index.html"), "w", encoding="utf-8", newline='') as f:
-		f.write(build_session_page(page, config["SESSION_PAGE"]))
+		f.write(build_session_page(page, config))
 	print("    Wrote Session")
-	with open(pathto("statistics", "index.html"), "w", encoding="utf-8") as f:
+	with open(pathto("statistics", "index.html"), "w", encoding="utf-8", newline='') as f:
-		f.write(build_statistics_page(page, articles))
+		f.write(build_statistics_page(config, page, articles))
 	print("    Wrote Statistics")
 
 	# Write auxiliary pages
-	if "PRINTABLE_FILE" in config and config["PRINTABLE_FILE"]:
+	if "SEARCHABLE_FILE" in config and config["SEARCHABLE_FILE"]:
-		with open(pathto(config["PRINTABLE_FILE"]), "w", encoding="utf-8") as f:
+		with open(pathto(config["SEARCHABLE_FILE"]), "w", encoding="utf-8", newline='') as f:
 			f.write(build_compiled_page(articles, config))
-		print("    Wrote compiled page to " + config["PRINTABLE_FILE"])
+		print("    Wrote compiled page to " + config["SEARCHABLE_FILE"])
 
-	with open(pathto("editor.html"), "w", encoding="utf-8") as f:
+	with open(pathto("editor.html"), "w", encoding="utf-8", newline='') as f:
 		editor = utils.load_resource("editor.html")
 		writtenArticles = ""
 		phantomArticles = ""

lexipython/resources/contents.html

@@ -0,0 +1,55 @@
+<script type="text/javascript">
+const order = {
+	INDEX: "index",
+	TURN: "turn",
+	PLAYER: "player",
+}
+var currentOrder = order.INDEX;
+setOrder = function(orderType)
+{
+	if (orderType == order.INDEX) {
+		document.getElementById("index-order").style.display = "block";
+		document.getElementById("turn-order").style.display = "none";
+		document.getElementById("player-order").style.display = "none";
+		document.getElementById("toggle-button").innerText = "Switch to turn order";
+		currentOrder = order.INDEX;
+	}
+	else if (orderType == order.TURN) {
+		document.getElementById("index-order").style.display = "none";
+		document.getElementById("turn-order").style.display = "block";
+		document.getElementById("player-order").style.display = "none";
+		document.getElementById("toggle-button").innerText = "Switch to player order";
+		currentOrder = order.TURN;
+	}
+	else if (orderType == order.PLAYER) {
+		document.getElementById("index-order").style.display = "none";
+		document.getElementById("turn-order").style.display = "none";
+		document.getElementById("player-order").style.display = "block";
+		document.getElementById("toggle-button").innerText = "Switch to index order";
+		currentOrder = order.PLAYER;
+	}
+}
+contentsToggle = function() {
+	if (currentOrder == order.INDEX)
+		setOrder(order.TURN);
+	else if (currentOrder == order.TURN)
+		setOrder(order.PLAYER);
+	else if (currentOrder == order.PLAYER)
+		setOrder(order.INDEX);
+}
+window.onload = function(){
+	if (location.search.search("byindex") > 0)
+	{
+		setOrder(order.INDEX);
+	}
+	if (location.search.search("byturn") > 0)
+	{
+		setOrder(order.TURN);
+	}
+	if (location.search.search("byplayer") > 0)
+	{
+		setOrder(order.PLAYER);
+	}
+}
+</script>
+<button id="toggle-button" onClick="javascript:contentsToggle()">Switch to turn order</button>

lexipython/resources/lexicon.cfg

@@ -0,0 +1,86 @@
+# LEXIPYTHON CONFIG FILE
+#
+# This file defines the configuration values for an instance of Lexipython.
+
+# The title of the Lexicon game, displayed at the top of each entry.
+>>>LEXICON_TITLE>>>
+Lexicon Title
+<<<LEXICON_TITLE<<<
+
+# The sidebar image. Constrained to 140-px.
+>>>LOGO_FILENAME>>>
+logo.png
+<<<LOGO_FILENAME<<<
+
+# The prompt for the Lexicon. Will be read as HTML and inserted into the
+# header directly.
+>>>PROMPT>>>
+<i>Prompt goes here</i>
+<<<PROMPT<<<
+
+# Session page content. Will be read as HTML and inserted into the body of
+# the session page directly.
+>>>SESSION_PAGE>>>
+<p>Put session information here, like the index grouping and turn count, where to send completed entries, index assignments, turn schedule, and so on.</p>
+<<<SESSION_PAGE<<<
+
+# Index headers.
+# An index header is declared as id:pattern or id[pri]:pattern. An article is
+#   sorted under the first index it matches. Matches are checked in descending
+#   order of pri, and in list order for indices of equal pri. An undefined pri
+#   value is 0. After matching is done, indices are written in list order
+#   regardless of pri. Index patterns must be unique, regardless of index type.
+# A character index has id "char". An article matches a character index if the
+#   first letter of its title is one of the characters in the index's pattern.
+# A prefix index has id "prefix". An article matches a prefix index if the
+#   title begins with the pattern.
+# The etc index has id "etc". An article always matches the etc index. The
+#   pattern is used as the index display name.
+>>>INDEX_LIST>>>
+char:ABC
+char:DEF
+char:GHI
+char:JKL
+char:MNO
+char:PQRS
+char:TUV
+char:WXYZ
+etc:&c.
+<<<INDEX_LIST<<<
+
+# Toggles and order for whichs tatistics to display.
+# Pagerank-based statistics require networkx to be installed.
+>>>STATISTICS>>>
+top_pagerank           on
+most_citations_made    on
+most_citations_to      on
+longest_article        on
+cumulative_wordcount   off
+player_pagerank        on
+player_citations_made  on
+player_citations_to    on
+bottom_pagerank        off
+undercited             off
+<<<STATISTICS<<<
+
+# The default sorting to use on the contents page.
+# Allowed values are "index", "turn", and "player"
+>>>DEFAULT_SORT>>>
+index
+<<<DEFAULT_SORT<<<
+
+# Flag to enable addendum articles. If enabled, articles with the same title
+# and a later turn than another article will be appended to that article.
+>>>ALLOW_ADDENDA>>>
+False
+<<<ALLOW_ADDENDA<<<
+
+# Graphviz file name. If present, the graph of page citations will be written
+# in the .dot format.
+>>>GRAPHVIZ_FILE>>>
+<<<GRAPHVIZ_FILE<<<
+
+# Searchable version file name. If present, the lexicon will be compiled and
+# written into a single, easily-searchable HTML file.
+>>>SEARCHABLE_FILE>>>
+<<<SEARCHABLE_FILE<<<

lexipython/resources/lexicon.css

@@ -0,0 +1,117 @@
+body {
+	background-color: #eeeeee;
+	line-height: 1.4;
+	font-size: 16px;
+}
+div#wrapper {
+	max-width: 800px;
+	position: absolute;
+	left: 0;
+	right: 0;
+	margin: 0 auto;
+}
+div#header {
+	padding: 5px;
+	margin: 5px;
+	background-color: #ffffff;
+	box-shadow: 2px 2px 10px #888888;
+	border-radius: 5px;
+}
+div#header p, div#header h2 {
+	margin: 5px;
+}
+div#sidebar {
+	width: 200px;
+	float:left;
+	margin:5px;
+	padding: 8px;
+	text-align: center;
+	background-color: #ffffff;
+	box-shadow: 2px 2px 10px #888888;
+	border-radius: 5px;
+}
+img#logo {
+	max-width: 200px;
+}
+table {
+	table-layout: fixed;
+	width: 100%;
+}
+div#sidebar table {
+	border-collapse: collapse;
+}
+div.citeblock table td:first-child + td a {
+	justify-content: flex-end;
+}
+div.misclinks table td a {
+	justify-content: center;
+}
+table a {
+	display: flex;
+	padding: 3px;
+	background-color: #dddddd;
+	border-radius: 5px;
+	text-decoration: none;
+}
+div#sidebar table a {
+	justify-content: center;
+}
+table a:hover {
+	background-color: #cccccc;
+}
+div#sidebar table td {
+	padding: 0px; margin: 3px 0;
+	border-bottom: 8px solid transparent;
+}
+div#content {
+	position: absolute;
+	right: 0px;
+	left: 226px;
+	max-width: 564px;
+	margin: 5px;
+}
+div.contentblock {
+	background-color: #ffffff;
+	box-shadow: 2px 2px 10px #888888;
+	margin-bottom: 5px;
+	padding: 10px;
+	width: auto;
+	border-radius: 5px;
+}
+div.contentblock h3 {
+	margin: 0.3em 0;
+}
+a.phantom {
+	color: #cc2200;
+}
+div.citeblock a.phantom {
+	font-style: italic;
+}
+span.signature {
+	text-align: right;
+}
+@media only screen and (max-width: 816px) {
+	div#wrapper {
+		padding: 5px;
+	}
+	div#header {
+		max-width: 554;
+		margin: 0 auto;
+	}
+	div#sidebar {
+		max-width: 548;
+		width: inherit;
+		float: inherit;
+		margin: 5px auto;
+	}
+	div#content {
+		max-width: 564;
+		position: static;
+		right: inherit;
+		margin: 5px auto;
+	}
+	img#logo {
+		max-width: inherit;
+		width: 100%;
+	}
+}

lexipython/resources/rules.html

@@ -55,7 +55,7 @@
 <li>As the game goes on, it may come to pass that a player must write an
 	article in an index, but that index is full, and that player has already
 	cited all the phantoms in it. When this happens, the player instead writes
-	their article as **Ersatz Scrivener**, radical skeptic. Ersatz does not
+	their article as <b>Ersatz Scrivener</b>, radical skeptic. Ersatz does not
 	believe in the existence of whatever he is writing about, no matter how
 	obvious it seems to others or how central it is in the developing history
 	of the world. For Ersatz, all references, testimony, etc. with regard to

lexipython/statistics.py

@@ -0,0 +1,317 @@
+# Third party imports
+try:
+	import networkx # For pagerank analytics
+	NETWORKX_ENABLED = True
+except:
+	NETWORKX_ENABLED = False
+
+# Application imports
+from utils import titlesort
+
+
+def reverse_statistics_dict(stats, reverse=True):
+	"""
+	Transforms a dictionary mapping titles to a value into a list of values
+	and lists of titles. The list is sorted by the value, and the titles are
+	sorted alphabetically.
+	"""
+	rev = {}
+	for key, value in stats.items():
+		if value not in rev:
+			rev[value] = []
+		rev[value].append(key)
+	for key, value in rev.items():
+		rev[key] = sorted(value, key=lambda t: titlesort(t))
+	return sorted(rev.items(), key=lambda x:x[0], reverse=reverse)
+
+
+def itemize(stats_list):
+	"""
+	Formats a list consisting of tuples of ranks and lists of ranked items.
+	"""
+	return map(lambda x: "{0} – {1}".format(x[0], "; ".join(x[1])), stats_list)
+
+
+class LexiconStatistics():
+	"""
+	A wrapper for a persistent statistics context with some precomputed
+	values around for convenience.
+
+	The existence of addendum articles complicates how some statistics are
+	computed. An addendum is an article, with its own author, body, and
+	citations, but in a Lexicon it exists appended to another article. To handle
+	this, we distinguish an _article_ from a _page_. An article is a unit parsed
+	from a single source file. A page is a main article and all addendums under
+	the same title.
+	"""
+
+	def __init__(self, articles):
+		self.articles = articles
+		self.min_turn = 0
+		self.max_turn = 0
+		self.players = set()
+		self.title_to_article = {}
+		self.title_to_page = {}
+		self.stat_block = "<div class=\"contentblock\"><u>{0}</u><br>{1}</div>\n"
+		# Pagerank may not be computable if networkx isn't installed.
+		self.title_to_pagerank = None
+
+		for main_article in articles:
+			page_title = main_article.title
+			self.title_to_page[page_title] = [main_article]
+			self.title_to_page[page_title].extend(main_article.addendums)
+			for article in self.title_to_page[page_title]:
+				# Disambiguate articles by appending turn number to the title
+				key = "{0.title} (T{0.turn})".format(article)
+				self.title_to_article[key] = article
+				if article.player is not None:
+					# Phantoms have turn MAXINT by convention
+					self.min_turn = min(self.min_turn, article.turn)
+					self.max_turn = max(self.max_turn, article.turn)
+					self.players.add(article.player)
+
+	def _try_populate_pagerank(self):
+		"""Computes pagerank if networkx is imported."""
+		if NETWORKX_ENABLED and self.title_to_pagerank is None:
+			# Create a citation graph linking page titles.
+			G = networkx.Graph()
+			for page_title, articles in self.title_to_page.items():
+				for article in articles:
+					for citation in article.citations:
+						G.add_edge(page_title, citation.target)
+
+			# Compute pagerank on the page citation graph.
+			self.title_to_pagerank = networkx.pagerank(G)
+			# Any article with no links in the citation graph have no pagerank.
+			# Assign these pagerank 0 to avoid key errors or missing pages in
+			# the stats.
+			for page_title, articles in self.title_to_page.items():
+				if page_title not in self.title_to_pagerank:
+					self.title_to_pagerank[page_title] = 0
+
+	def stat_top_pagerank(self):
+		"""Computes the top 10 pages by pagerank."""
+		self._try_populate_pagerank()
+
+		if not self.title_to_pagerank:
+			# If networkx was not successfully imported, skip the pagerank.
+			top_ranked_items = "networkx must be installed to compute pageranks."
+
+		else:
+			# Get the top ten articles by pagerank.
+			top_pageranks = reverse_statistics_dict(self.title_to_pagerank)[:10]
+			# Replace the pageranks with ordinals.
+			top_ranked = enumerate(map(lambda x: x[1], top_pageranks), start=1)
+			# Format the ranks into strings.
+			top_ranked_items = itemize(top_ranked)
+
+		# Format the statistics block.
+		return self.stat_block.format(
+			"Top 10 articles by page rank:",
+			"<br>".join(top_ranked_items))
+
+	def stat_most_citations_made(self):
+		"""Computes the top 3 ranks for citations made FROM a page."""
+		# Determine which pages are cited from all articles on a page.
+		pages_cited = {
+			page_title: set()
+			for page_title in self.title_to_page.keys()}
+		for page_title, articles in self.title_to_page.items():
+			for article in articles:
+				for citation in article.citations:
+					pages_cited[page_title].add(citation.target)
+		# Compute the number of unique articles cited by a page.
+		for page_title, cite_titles in pages_cited.items():
+			pages_cited[page_title] = len(cite_titles)
+
+		# Reverse and itemize the citation counts.
+		top_citations = reverse_statistics_dict(pages_cited)[:3]
+		top_citations_items = itemize(top_citations)
+
+		# Format the statistics block.
+		return self.stat_block.format(
+			"Cited the most pages:",
+			"<br>".join(top_citations_items))
+
+	def stat_most_citations_to(self):
+		"""Computes the top 3 ranks for citations made TO a page."""
+		# Determine which pages cite a page.
+		pages_cited_by = {
+			page_title: set()
+			for page_title in self.title_to_page.keys()}
+		for page_title, articles in self.title_to_page.items():
+			for article in articles:
+				for citation in article.citations:
+					pages_cited_by[citation.target].add(page_title)
+		# Compute the number of unique articles that cite a page.
+		for page_title, cite_titles in pages_cited_by.items():
+			pages_cited_by[page_title] = len(cite_titles)
+
+		# Reverse and itemize the citation counts.
+		top_cited = reverse_statistics_dict(pages_cited_by)[:3]
+		top_cited_items = itemize(top_cited)
+
+		# Format the statistics block.
+		return self.stat_block.format(
+			"Cited by the most pages:",
+			"<br>".join(top_cited_items))
+
+	def stat_longest_article(self):
+		"""Computes the top 3 longest articles."""
+		# Compute the length of each article (not page).
+		title_to_article_length = {}
+		for article_title, article in self.title_to_article.items():
+			# Write all citation aliases into the article text to accurately
+			# compute word count as written.
+			format_map = {
+				"c"+str(c.id): c.text
+				for c in article.citations
+			}
+			plain_content = article.content.format(**format_map)
+			word_count = len(plain_content.split())
+			title_to_article_length[article_title] = word_count
+
+		# Reverse and itemize the article lengths.
+		top_length = reverse_statistics_dict(title_to_article_length)[:3]
+		top_length_items = itemize(top_length)
+
+		# Format the statistics block.
+		return self.stat_block.format(
+			"Longest articles:",
+			"<br>".join(top_length_items))
+
+	def stat_cumulative_wordcount(self):
+		"""Computes the cumulative word count of the lexicon."""
+		# Initialize all extant turns to 0.
+		turn_to_cumulative_wordcount = {
+			turn_num: 0
+			for turn_num in range(self.min_turn, self.max_turn + 1)
+		}
+		for article_title, article in self.title_to_article.items():
+			# Compute each article's word count.
+			format_map = {
+				"c"+str(c.id): c.text
+				for c in article.citations
+			}
+			plain_content = article.content.format(**format_map)
+			word_count = len(plain_content.split())
+			# Add the word count to each turn the article exists in.
+			for turn_num in range(self.min_turn, self.max_turn + 1):
+				if article.turn <= turn_num:
+					turn_to_cumulative_wordcount[turn_num] += word_count
+
+		# Format the statistics block.
+		len_list = [(str(k), [str(v)]) for k,v in turn_to_cumulative_wordcount.items()]
+		return self.stat_block.format(
+			"Aggregate word count by turn:",
+			"<br>".join(itemize(len_list)))
+
+	def stat_player_pagerank(self):
+		"""Computes each player's share of the lexicon's pagerank scores."""
+		self._try_populate_pagerank()
+
+		if not self.title_to_pagerank:
+			# If networkx was not successfully imported, skip the pagerank.
+			player_rank_items = "networkx must be installed to compute pageranks."
+
+		else:
+			player_to_pagerank = {
+				player: 0
+				for player in self.players}
+			# Accumulate page pagerank to the main article's author.
+			for page_title, articles in self.title_to_page.items():
+				page_author = articles[0].player
+				if page_author is not None:
+					player_to_pagerank[page_author] += self.title_to_pagerank[page_title]
+			# Round pageranks off to 3 decimal places.
+			for player, pagerank in player_to_pagerank.items():
+				player_to_pagerank[player] = round(pagerank, 3)
+
+			# Reverse and itemize the aggregated pageranks.
+			player_rank = reverse_statistics_dict(player_to_pagerank)
+			player_rank_items = itemize(player_rank)
+
+		# Format the statistics block.
+		return self.stat_block.format(
+			"Player aggregate page rank:",
+			"<br>".join(player_rank_items))
+
+	def stat_player_citations_made(self):
+		"""Computes the total number of citations made BY each player."""
+		pages_cited_by_player = {
+			player: 0
+			for player in self.players}
+		# Add the number of citations from each authored article (not page).
+		for article_title, article in self.title_to_article.items():
+			if article.player is not None:
+				pages_cited_by_player[article.player] += len(article.citations)
+
+		# Reverse and itemize the counts.
+		player_cites_made_ranks = reverse_statistics_dict(pages_cited_by_player)
+		player_cites_made_items = itemize(player_cites_made_ranks)
+
+		# Format the statistics block.
+		return self.stat_block.format(
+			"Citations made by player:",
+			"<br>".join(player_cites_made_items))
+
+	def stat_player_citations_to(self):
+		"""Computes the total number of citations made TO each player's
+		authored pages."""
+		pages_cited_by_by_player = {
+			player: 0
+			for player in self.players}
+		# Add the number of citations made to each page (not article).
+		for page_title, articles in self.title_to_page.items():
+			page_author = articles[0].player
+			if page_author is not None:
+				pages_cited_by_by_player[page_author] += len(articles[0].citedby)
+
+		# Reverse and itemize the results.
+		cited_times_ranked = reverse_statistics_dict(pages_cited_by_by_player)
+		cited_times_items = itemize(cited_times_ranked)
+
+		# Format the statistics block.
+		return self.stat_block.format(
+			"Citations made to article by player:",
+			"<br>".join(cited_times_items))
+
+	def stat_bottom_pagerank(self):
+		"""Computes the bottom 10 pages by pagerank."""
+		self._try_populate_pagerank()
+
+		if not self.title_to_pagerank:
+			# If networkx was not successfully imported, skip the pagerank.
+			bot_ranked_items = "networkx must be installed to compute pageranks."
+
+		else:
+			# Phantoms have no pagerank, because they don't cite anything.
+			exclude = [
+				a.title
+				for a in self.articles
+				if a.player is None]
+			rank_by_written_only = {
+				k:v
+				for k,v in self.title_to_pagerank.items()
+				if k not in exclude}
+			
+			# Reverse, enumerate, and itemize the bottom 10 by pagerank.
+			pageranks = reverse_statistics_dict(rank_by_written_only)
+			bot_ranked = list(enumerate(map(lambda x: x[1], pageranks), start=1))[-10:]
+			bot_ranked_items = itemize(bot_ranked)
+
+		# Format the statistics block.
+		return self.stat_block.format(
+			"Bottom 10 articles by page rank:",
+			"<br>".join(bot_ranked_items))
+
+	def stat_undercited(self):
+		"""Computes which articles have 0 or 1 citations made to them."""
+		undercited = {
+			page_title: len(articles[0].citedby)
+			for page_title, articles in self.title_to_page.items()
+			if len(articles[0].citedby) < 2}
+		undercited_items = itemize(reverse_statistics_dict(undercited))
+		return self.stat_block.format(
+			"Undercited articles:",
+			"<br>".join(undercited_items))

lexipython/utils.py

@@ -0,0 +1,87 @@
+import os
+import re
+import io
+from urllib import parse
+import pkg_resources
+
+# Short utility functions for handling titles
+
+def titlecase(s):
+	"""
+	Capitalizes the first word.
+	"""
+	s = s.strip()
+	return s[:1].capitalize() + s[1:]
+
+def titleescape(s):
+	"""
+	Makes an article title filename-safe.
+	"""
+	s = s.strip()
+	s = re.sub(r"\s+", '_', s)  # Replace whitespace with _
+	s = re.sub(r"~", '-', s)    # parse.quote doesn't catch ~
+	s = parse.quote(s)          # Encode all other characters
+	s = re.sub(r"%", "", s)     # Strip encoding %s
+	s = s[:64]                  # Limit to 64 characters
+	return s
+
+def titlesort(s):
+	"""
+	Reduces titles down for sorting.
+	"""
+	s = s.lower()
+	if s.startswith("the "): return s[4:]
+	if s.startswith("an "): return s[3:]
+	if s.startswith("a "): return s[2:]
+	return s
+
+# Load functions
+
+def load_resource(filename, cache={}):
+	"""Loads files from the resources directory with caching."""
+	if filename not in cache:
+		binary = pkg_resources.resource_string("resources", filename)
+		unistr = binary.decode("utf-8")
+		cache[filename] = unistr
+	return cache[filename]
+
+def parse_config_file(f):
+	"""Parses a Lexipython config file."""
+	config = {}
+	line = f.readline()
+	while line:
+		# Skim lines until a value definition begins
+		conf_match = re.match(r">>>([^>]+)>>>\s+", line)
+		if not conf_match:
+			line = f.readline()
+			continue
+		# Accumulate the conf value until the value ends
+		conf = conf_match.group(1)
+		conf_value = ""
+		line = f.readline()
+		conf_match = re.match(r"<<<{0}<<<\s+".format(conf), line)
+		while line and not conf_match:
+			conf_value += line
+			line = f.readline()
+			conf_match = re.match(r"<<<{0}<<<\s+".format(conf), line)
+		if not line:
+			raise EOFError("Reached EOF while reading config value {}".format(conf))
+		config[conf] = conf_value.strip()
+	return config
+
+def load_config(name):
+	"""
+	Loads values from a Lexicon's config file.
+	"""
+	with open(os.path.join("lexicon", name, "lexicon.cfg"), "r", encoding="utf8") as f:
+		config = parse_config_file(f)
+	# Check that no values are missing that are present in the default config
+	with io.StringIO(load_resource("lexicon.cfg")) as f:
+		default_config = parse_config_file(f)
+	missing_keys = []
+	for key in default_config.keys():
+		if key not in config:
+			missing_keys.append(key)
+	if missing_keys:
+		raise KeyError("{} missing config values for: {}".format(name, " ".join(missing_keys)))
+	return config

src/resources/contents.html

@@ -1,29 +0,0 @@
-<script type="text/javascript">
-contentsToggle = function() {
-	var b = document.getElementById("toggle-button");
-	var i = document.getElementById("index-order");
-	var t = document.getElementById("turn-order");
-	if (t.style.display == "none") {
-		i.style.display = "none";
-		t.style.display = "block";
-		b.innerText = "Switch to index order";
-	} else {
-		i.style.display = "block";
-		t.style.display = "none";
-		b.innerText = "Switch to turn order";
-	}
-}
-window.onload = function(){
-	if (location.search.search("byturn") > 0)
-	{
-		document.getElementById("turn-order").style.display = "block";
-		document.getElementById("toggle-button").innerText = "Switch to index order";
-	}
-	if (location.search.search("byindex") > 0)
-	{
-		document.getElementById("index-order").style.display = "block";
-		document.getElementById("toggle-button").innerText = "Switch to turn order";
-	}
-}
-</script>
-<button id="toggle-button" onClick="javascript:contentsToggle()">Switch to turn order</button>

src/resources/lexicon.cfg

@@ -1,61 +0,0 @@
-# LEXIPYTHON CONFIG FILE
-#
-# This file defines the configuration values for an instance of Lexipython.
-# Configuration values are written as:
->>>CONFIG_NAME>>>
-value
-<<<CONFIG_NAME<<<
-# The above defines a config value named CONFIG_NAME with a value of "value".
-
-# The title of the Lexicon game, displayed at the top of each entry.
->>>LEXICON_TITLE>>>
-Lexicon Title
-<<<LEXICON_TITLE<<<
-
-# The sidebar image. Constrained to 140-px.
->>>LOGO_FILENAME>>>
-logo.png
-<<<LOGO_FILENAME<<<
-
-# The prompt for the Lexicon. Will be read as HTML and inserted into the
-# header directly.
->>>PROMPT>>>
-<i>Prompt goes here</i>
-<<<PROMPT<<<
-
-# Session page content. Will be read as HTML and inserted into the body of
-# the session page directly.
->>>SESSION_PAGE>>>
-<p>Put session information here, like the index grouping and turn count, where to send completed entries, index assignments, turn schedule, and so on.</p>
-<<<SESSION_PAGE<<<
-
-# Index headers. An index is a string of characters, which are the characters
-# an entry has to begin with to fall under that index. Indices will be listed
-# in the order written. Entries will be put into the first index they match.
-# Leftover entries will be listed under "&c." at the end.
->>>INDEX_LIST>>>
-ABC
-DEF
-GHI
-JKL
-MNO
-PQRS
-TUV
-WXYZ
-<<<INDEX_LIST<<<
-
-# The default sorting to use on the contents page.
-# Allowed values are "?byturn" and "?byindex".
->>>DEFAULT_SORT>>>
-?byturn
-<<<DEFAULT_SORT<<<
-
-# Graphviz file name. If present, the graph of page citations will be written
-# in the dot file format.
->>>GRAPHVIZ_FILE>>>
-<<<GRAPHVIZ_FILE<<<
-
-# Print version file name. If present, the lexicon will be compiled and written
-# into a single print-ready HTML file.
->>>PRINTABLE_FILE>>>
-<<<PRINTABLE_FILE<<<

src/resources/lexicon.css

@@ -1,36 +0,0 @@
-body { background-color: #eeeeee; line-height: 1.4; font-size: 16px; }
-div#wrapper { max-width: 800px; position: absolute; left: 0; right: 0;
-	margin: 0 auto; }
-div#header { padding: 5px; margin: 5px; background-color: #ffffff;
-	box-shadow: 2px 2px 10px #888888; border-radius: 5px; }
-div#header p, div#header h2 { margin: 5px; }
-div#sidebar { width: 200px; float:left; margin:5px; padding: 8px;
-	text-align: center; background-color: #ffffff;
-	box-shadow: 2px 2px 10px #888888; border-radius: 5px; }
-img#logo { max-width: 200px; }
-table { table-layout: fixed; width: 100%; }
-div#sidebar table { border-collapse: collapse; }
-div.citeblock table td:first-child + td a { justify-content: flex-end; }
-table a { display: flex; padding: 3px; background-color: #dddddd;
-	border-radius: 5px; text-decoration: none; }
-div#sidebar table a { justify-content: center; }
-table a:hover { background-color: #cccccc; }
-div#sidebar table td { padding: 0px; margin: 3px 0;
-	border-bottom: 8px solid transparent; }
-div#content { position: absolute; right: 0px; left: 226px; max-width: 564px;
-	margin: 5px; }
-div.contentblock { background-color: #ffffff; box-shadow: 2px 2px 10px #888888;
-	margin-bottom: 5px; padding: 10px; width: auto; border-radius: 5px; }
-div.contentblock h3 { margin: 0.3em 0; }
-a.phantom { color: #cc2200; }
-div.citeblock a.phantom { font-style: italic; }
-span.signature { text-align: right; }
-@media only screen and (max-width: 816px) {
-	div#wrapper { padding: 5px; }
-	div#header { max-width: 554; margin: 0 auto; }
-	div#sidebar { max-width: 548; width: inherit; float: inherit;
-		margin: 5px auto; }
-	div#content { max-width: 564; position: static; right: inherit;
-		margin: 5px auto; }
-	img#logo { max-width: inherit; width: 100%; }
-}

src/utils.py

@@ -1,75 +0,0 @@
-import os
-import re
-from urllib import parse
-
-# Short utility functions for handling titles
-
-def titlecase(s):
-	"""
-	Capitalizes the first word.
-	"""
-	s = s.strip()
-	return s[:1].capitalize() + s[1:]
-
-def titleescape(s):
-	"""
-	Makes an article title filename-safe.
-	"""
-	s = s.strip()
-	s = re.sub(r"\s+", '_', s)  # Replace whitespace with _
-	s = parse.quote(s)          # Encode all other characters
-	s = re.sub(r"%", "", s)     # Strip encoding %s
-	s = s[:64]                  # Limit to 64 characters
-	return s
-
-def titlesort(s):
-	"""
-	Reduces titles down for sorting.
-	"""
-	s = s.lower()
-	if s.startswith("the "): return s[4:]
-	if s.startswith("an "): return s[3:]
-	if s.startswith("a "): return s[2:]
-	return s
-
-# Load functions
-
-def load_resource(filename, cache={}):
-	"""Loads files from the resources directory with caching."""
-	if filename not in cache:
-		with open(os.path.join("src", "resources", filename), "r", encoding="utf-8") as f:
-			cache[filename] = f.read()
-	return cache[filename]
-
-def load_config(name):
-	"""
-	Loads values from a Lexicon's config file.
-	"""
-	config = {}
-	with open(os.path.join("lexicon", name, "lexicon.cfg"), "r", encoding="utf8") as f:
-		line = f.readline()
-		while line:
-			# Skim lines until a value definition begins
-			conf_match = re.match(r">>>([^>]+)>>>\s+", line)
-			if not conf_match:
-				line = f.readline()
-				continue
-			# Accumulate the conf value until the value ends
-			conf = conf_match.group(1)
-			conf_value = ""
-			line = f.readline()
-			conf_match = re.match(r"<<<{0}<<<\s+".format(conf), line)
-			while line and not conf_match:
-				conf_value += line
-				line = f.readline()
-				conf_match = re.match(r"<<<{0}<<<\s+".format(conf), line)
-			if not line:
-				# TODO Not this
-				raise SystemExit("Reached EOF while reading config value {}".format(conf))
-			config[conf] = conf_value.strip()
-	# Check that all necessary values were configured
-	for config_value in ['LEXICON_TITLE', 'PROMPT', 'SESSION_PAGE', "INDEX_LIST"]:
-		if config_value not in config:
-			# TODO Not this either
-			raise SystemExit("Error: {} not set in lexipython.cfg".format(config_value))
-	return config