This fixes a private source easier than solving the problem of sources introducing dependencies |
||
---|---|---|
inquisitor | ||
sources | ||
.gitignore | ||
README.md | ||
default.nix | ||
flake.lock | ||
flake.nix | ||
poetry.lock | ||
pyproject.toml | ||
shell.nix |
README.md
Inquisitor
Inquisitor is a content aggregator for personal use. Unlike RSS aggregators, it is not restricted to content feeds of a particular format. Instead, feed items are generated by user-provided arbitrary Python snippets.
Producing items
Items are created and added to the Inquisitor feed with the update
command.
inquisitor update provider1:argument provider2 ...
Source API for Inquisitor
The update
command searches the source directory for source providers. The source directory is specified by --srcdir
and defaults to ./sources
. A source provider is any file in the source directory that ends in .py
, validly imports as a Python 3 module, and provides a get_sources
function.
The get_sources
function implemented by a source provider should take one argument, a string. When a provider is specified as provider:argument
, Inquisitor passes the string after the colon to that provider's get_sources
when invoking it. Otherwise, the value is None
. get_sources
should return an iterable of sources. Inquisitor will execute get_sources
on each source provider specified in the update
invocation.
A source is a dictionary with two keyvalues. 'name'
should map to a string containing the name of the source. 'fetch_new'
should map to a function. The fetch_new
function takes one argument, a state dictionary. This is used for caching a persistent state between invocations. It should return a list of items. Inquisitor will execute fetch_new
for each source returned by a get_sources
call, add the new items to the source's folder, and then clean out old, inactive items from that source.
Items are distinguished by their id
field, which becomes the item's filename in the source's item folder. If an item already exists from a previous fetch_new
call from that source, the new values for any fields will overwrite the old ones, except for the active
and created
fields.
Inquisitor API for sources
When a source provider is imported or a function from a provider or source is executed, the current working directory is the source provider directory.
The builtin Item()
is provided for creating items. id
and title
are required parameters. Item()
sets the item's source
, created
, and active
fields automatically. The other item fields (link
, ts
, author
, body
, and tags
) can be specified as keyword arguments to the builtin or set by the corresponding field on the returned Item
object.
Consuming items
inquisitor run
launches the built-in Flask server. The Flask development server is insecure and shouldn't be publicly exposed, but Inquisitor is for personal use, so it probably shouldn't be publicly exposed in the first place. You should either run the Inquisitor server on the same computer you plan to use for reading the feed or put it behind a reverse proxy only accessible to your local network.
The Inquisitor feed consists of items. Each item has a title. Items with a url
field will have a link to open the URL in a new tab. All items have a button to mark them as read. Items marked as read will fade out. Items with a body
field will expand to display the body when the title is clicked.
Feed logic
A source produces a list of items when called. Each of these items is written to a file and displayed in the feed. When an item is marked as read, its active
field is marked as false. This is to prevent the item from reappearing in the feed if the source produces it the next time it is called. Conversely, an item continues to appear in the feed if it is still marked as active, even if its source no longer returns it. This ensures that all items from a source are shown to the user. An item is only removed from the source's folder once it is both inactive and no longer appears in the source
Some feeds may return an item, not return it later, then return it again. Normally, this would result in the same item appearing in the feed after having been deactivated. To prevent this, set the ttl
field on an item with a timestamp. The item will not be cleaned up until the current time postdates the ttl
value.