Mynij - search faster, offline

Mynij Milestone 7: Catalog of selected sources

Mynij Store has Sitemaps source for various websites. They are available for install in Mynij PWA in one click. No login required.
  • Last Update:2021-11-24
  • Version:001
  • Language:en

The goal of this Milestone is to provide a centralized platform where sitemaps sources will be stored so that Mynij users can bootstrap quickly a personal search engine with their preferred selection of sources.

 

Mynij store

Mynij source store is a version Officejs store which was configured for Mynij. The OfficeJS appstore is a software that already hosts an ecosystem of progressive web applications developed with renderJS and jIO frameworks among others.

Mynij source store provides a platform for hosting and curating JSON sources that were previously exported from mynij PWA. It provides a simplified way to quickly configure Mynij search engine.

For users willing to host their own Store, follow this documentation on how to request ERP5 instance using a webrunner, then install officejs Software Release with the additional erp5_mynij_appstore business template which include all customisations ùade for Mynij. 

The public instance of Mynij source store where we share some sample sources can be reached at https://store.mynij.com. No login or account is required to download sources. The home page lists all published and accepted sources that are available. Users can filter sources by typing search patterns in the search bar.

A click on a source opens Mynij PWA, downloads data from url parameter, then automatically installs and builds the source index. Each source in the store has an installation URL which looks like this:

https://mynij.com/#/?page=load_source&url=https://source.mynij.com/osp-XX/REVISION_HASH/Nexedi.com.json

If the source JSON contains torrent infohash, mynij will first try to download and install index data from Webtorrent. If there is no torrent available for download or torrent is not downloadable, the source index will be built by Mynij PWA by crawling all content.

Exporting sources from Mynij app

"Export Sources" generates a JSON File containing the source definition (title, description, links, logo base64, torrent information). This file can then published to Mynij store and can be later used to bootstrap Mynij search engine.


 

Exported JSON containing a list of sources looks like the content bellow:

[{
    "title": "Nexedi.com",
    "description": "All Nexedi SA website sitemaps",
    "links": "https://softinst140630.host.vifib.net/www.nexedi.com.xml\nhttps://softinst140630.host.vifib.net/stack.nexedi.com.xml",
    "logo":{
            "url":"data:image/webp;base64,UklGRuBe....",
            "file_name":"nexedi-partner-logo.png"
    }
}]

 

Contribute a source

Publishing a source on mynij store is possible with https://publisher.mynij.com/ which is a part of the Officejs AppStore software customised for Mynij called "Mynij Publisher". Users can publish JSON Sources exported from the PWA using Mynij publisher. An account is required to publish a source. To request an account and contribute a source, please send a mail to contributor@mynij.com.  Once logged in, the user can click on the "Add" button to submit a new source.

The next page in the image below shows a form to create the source and upload a JSON file that was exported from Mynij PWA application. We submit the form by selecting the file and then clicking on New Source button. 

As soon the state of the new source changed to Submitted, we can use the Test Url associated to the source for testing the source from within Mynij PWA. We can then ensure that everything is fine before effective publication of the source.

At this stage, the source is not yet visible inside the public store. To publish it, we mist click on the Ask Publication button to issue a publication request. The request will be reviewed by a Source Store administrator and will be accepted or rejected depending on whether the source is considered as acceptable or not. This is the curation process.

All sources that are accepted are published and available at https://store.mynij.com.

Update published source

Updating a source is useful to change some information in the JSON of the source: additional entries, modified entries, broken URL, etc. For this to happen, the user should first add a new Release to the source "Product". The notion of "Product" in the source store is intended to group all versions of the same source and track their release.

In order to update a source, the user should first select the existing source on https://publish.mynij.com. Then he or she should click on Add button to add a new release. Next step is to upload the JSON source file, edit the change log and submit the new version for effective release.

The validation process is the same as for adding a new source.