Difference between revisions of "Free Software Directory:Infrastructure"

From Free Software Directory
Jump to: navigation, search
(→‎Git: added feedback from bill-auger -- Thanks!)
(Added ===Public API=== -- Thanks Bone Baboon)
Line 54: Line 54:
 
programs including Firefox, Thunderbird, SeaMonkey, and Sunbird are
 
programs including Firefox, Thunderbird, SeaMonkey, and Sunbird are
 
trademarked and thus nonfree.
 
trademarked and thus nonfree.
 +
</pre>
 +
 +
==Data access==
 +
 +
===Public API===
 +
<pre>
 +
From: Bone Baboon
 +
Subject: Re: FSD as a Git repository
 +
Date: Sat, 17 Jul 2021 18:30:23 -0400
 +
Archive link: https://lists.gnu.org/archive/html/directory-discuss/2021-07/msg00035.html
 +
 +
Has the FSD's MediaWiki API been made publicly accessible?
 +
 +
If the FSF has not made the FSD's MediaWiki API publicly available
 +
yet then it would have to consider if it wants to make the API public
 +
and maintain it.
 +
 +
I asked some question in #mediawiki@libera about MediaWiki's API.  These
 +
links for the API documentation where shared:
 +
 +
<https://www.mediawiki.org/w/api.php>
 +
<https://www.mediawiki.org/wiki/API:Edit>
 +
<https://www.mediawiki.org/wiki/API:Main_page>
 +
 +
There is also an in development REST API that has "much less
 +
functionality":
 +
 +
<https://www.mediawiki.org/wiki/API:REST_API>
 +
 +
...
 +
 +
When I asked about MediaWiki API clients in #mediawiki@libera:
 +
* this link was shared <https://www.mediawiki.org/wiki/API:Client_code>
 +
* someone suggested pywikibot
 +
 +
...
 +
 +
When I asked about MediaWiki API clients in #mediawiki@libera:
 +
* this link was shared <https://www.mediawiki.org/wiki/API:Client_code>
 +
* someone suggested pywikibot
 
</pre>
 
</pre>
  

Revision as of 19:50, 17 July 2021

New anti-feature templates

This is an excellent first task for our new infrastructure.

The anti-feature text for all entries in https://directory.fsf.org/wiki/Free_Software_Directory:Antifeatures should not be part of the main page since we want to update the main page with pure .wiki files generated from Debian meta-data. Instead the anti-feature wiki text should be placed in special pages that are included in the main page.

Example for Adblock Plus, place this text in Antifeature:Adblock Plus. 

{{Antifeature
|Antifeature_description={{AttentionBox
 |title=Antifeature: Tracking comment
 |color=red
 |icon=<span style="font-size: 250%;">⚠</span>
 |message=Adblock Plus is ineffective for surveillance protection by default as it comes with ''Acceptable Ads'' enabled: These ads are not meant to be "ads that don't track you".
}}
|Antifeature_tags={{Project antifeature
|Antifeature=Adware
}}
{{Project antifeature
|Antifeature=Tracking
}}
}}

Data

See also Free_Software_Directory:Backlog_active#Import

Debian Packaging System

Disclaimer: The Debian Package Tracking System provides no software description needed for |Short description= and |Full description= in the entry pages.

The Debian Package Tracking System produces RDF metadata and is already included in DBpedia. For example:

External links

Repology

Import

"Repology monitors a huge number of package repositories and other sources comparing packages versions across them and gathering other information. Repology shows you in which repositories a given project is packaged, which version is the latest and which needs updating, who maintains the package, and other related information."

Export unique FSD entries

New repository: Free Software Directory

FreeAMO

http://git.savannah.gnu.org/cgit/directory.git/tree/subprojects/freeamo 

FreeAMO is a project used to get meta-data for free add-ons at
addons.mozilla.org (AMO) used to produce free add-on repositories for
free Mozilla-based programs.  contain nonfree add-ons and Mozilla
Foundation's addons.mozilla.org contains nonfree add-ons, and their
programs including Firefox, Thunderbird, SeaMonkey, and Sunbird are
trademarked and thus nonfree.

Data access

Public API

From: Bone Baboon
Subject: Re: FSD as a Git repository
Date: Sat, 17 Jul 2021 18:30:23 -0400
Archive link: https://lists.gnu.org/archive/html/directory-discuss/2021-07/msg00035.html

Has the FSD's MediaWiki API been made publicly accessible?

If the FSF has not made the FSD's MediaWiki API publicly available
yet then it would have to consider if it wants to make the API public
and maintain it.

I asked some question in #mediawiki@libera about MediaWiki's API.  These
links for the API documentation where shared:

<https://www.mediawiki.org/w/api.php>
<https://www.mediawiki.org/wiki/API:Edit>
<https://www.mediawiki.org/wiki/API:Main_page>

There is also an in development REST API that has "much less
functionality": 

<https://www.mediawiki.org/wiki/API:REST_API>

...

When I asked about MediaWiki API clients in #mediawiki@libera:
* this link was shared <https://www.mediawiki.org/wiki/API:Client_code>
* someone suggested pywikibot

...

When I asked about MediaWiki API clients in #mediawiki@libera:
* this link was shared <https://www.mediawiki.org/wiki/API:Client_code>
* someone suggested pywikibot

Data model

Git

From: Bone Baboon
To: directy-discuss@gnu.org
Subject: FSD as a Git repository
Date: 2021-07-12 22:41

I wanted to ask what people think about the idea of the FSD being a git
repository.  This idea is motivated by this email thread.
<https://mail.gnu.org/archive/html/directory-discuss/2021-06/msg00017.html>

Here is an example of how the FSD could work as a Git repository:

* The Git repository would have the normal readme, contributing, license
  files. 
* The FSD contents could be organized with a clear and deliberate
  directory hierarchy. 
* The directories and files would have clear and deliberate names.
* The files could be written in a simple plain text format (Markdown,
  org-mode, etc). 
* A static website of the Git repository and it's source code could be
  generated using free software tools like stagit and cgit. 
* The FSD's plain text files could be compiled to HTML and served as a
  static web site without JavaScript using free software (discount,
  org-mode, etc). 

There would be trade off between using the current MediaWiki setup and
using a Git setup.

Both Git and MediaWiki provide:
* Metadata on contributions
* History of changes

Git advantages:
* No graphical web browser required.
* No JavaScript required.
* A text web browser would be optional.
* A contributor does not need a computer with a graphical environment.
  Not all GPU's have free firmware. 
* A contributor would not have to sign up to be able to make
  contributions.  (signing up currently does not work without JavaScript
  and cookies) 
* The Git repository could be cloned and worked on offline.
* Free software tools could be used to work on the Git repository.
* Contributors can make edits with their free software text editor of
  choice. 
* Static websites without JavaScript for browsing the Git repository and
  source code would be accessible with text web browsers.
* Static websites without JavaScript rendered from the FSD's Git
  repository would be accessible to browse with a text web browsers.
* Once the content is in a plain text format it would be easier to move
  to another plain text format using free software (Pandoc for example)
  if that was desirable in the future.
* Simpler server administration without a database.
* Uses less system resources on the server.

Git disadvantages:
* Requires learning how to use Git.
* Requires learning a simple markdown format.
* Requires a data migration initiative to move the FSD's data from it's
  current MediaWiki format into plain text files.
* Requires individuals with commit access to the Git repository to
  accept patches submitted by contributors.

MediaWiki advantages:
* The FSD already uses MediaWiki.
* Currently works for contributors who have graphical browsers.
* Has forms that provide validation checks on entered data.
* Has reusable templates for common content.

MediaWiki disadvantages:
* Requires learning the how to use MediaWiki.
* Does not work well with text web browsers.  Can not sign in with EWW
  Emacs's built in web browser for example.  Not all GPU's have free
  firmware and are able to run a graphical web browser.

From: bill-auger
To: directy-discuss@gnu.org
Subject: FSD as a Git repository
Date: 2021-07-14 08:17

FWIW, mediawiki has a complete REST API - it can be accessed from
the command line with curl, etc - all of the discussion so far
has the regrettable presumption, that a web browser is required
to read/write data to/from the internet, or to do so in an
"accessible" way

API clients are easy to write; and there are clients made for
mediawiki already (weboob alone, has multiple plugins for
mediawiki) - AFAIAC, that would satisfy the text-only use-case,
just as well as gopher or anything else would, but without
replacing the wiki, migrating all of it's data, or even patching
any upstream code - the FSF would only need to enable API
access, if it is not already

if some web service has a complete remote API, that is already
the ideal option for text-only, no-js, accessibility, etc - an
API is fully accessible by its nature (no colors, no mouse
buttons to locate, etc)

in short, the best javscript is 'none', and the best web browser
is 'none' - no need to re-invent the wheel - mediawiki already
supports it

Wikibase

I propose Wikibase: wikidata.org uses Wikibase -- It goes _much_ faster wikidata.org than directory.fsf.org. (david hedlund)

Wikibase is a triplestore implementation. "Technical support: the data model should allow for adequate representations in existing data formats, e.g., JSON or RDF/OWL" - https://www.mediawiki.org/wiki/Wikibase/DataModel


my evaluation: Iank (talk) 10:40, 18 July 2018 (EDT)

Upsides:

It would give an api to update data. With semantic mediawiki, you have to update entire pages. This would allow better syncing from sources like mozilla and debian.

We could also use it to pull in data from wikidata, but that could also be done without adopting wikibase for our own data.

Downsides: wikipedia's use of it seems to be mostly around linking between languages, and a very small use of the actual data, and where it is used, it's not user friendly, the data magically appears from a template that that has a wikidata reference number and there is no easy way for users to navigate to that data to edit it.

https://en.wikipedia.org/wiki/Wikipedia:Wikidata

At this point it does not seem high priority to adopt it, but it's got potential if wikipedia creates a better user interface for it.

Page structure

#####################################################################
Import and export data

https://directory.fsf.org/wiki/<entry>
|
|--<entry>-debian.wiki -- generated by https://savannah.gnu.org/projects/directory
|		* This is a Debian package: Yes
|		* Package name
|		* Version
|		* Release date
|		* Dependencies (depends, recommends, suggests, enhances)
|--<entry>-user.wiki -- user contributions on https://directory.fsf.org/wiki/
		* Download link
		* OpenPGP signature URL
		* Donate
		* VCS Checkout
		* Categories
		* IRC channel
		* Etc

Example:

https://directory.fsf.org/wiki/Emacs
|
|--Emacs-debian.wiki
|		* This is a Debian package: Yes
|		* Package name: Emacs
|		* Version: 26.1
|		* Release date: 28 May 2018
|		* Dependencies: <will be listed here>
|--Emacs-user.wiki
		* Download link: http://ftpmirror.gnu.org/emacs/
		* VCS Checkout: git clone git://git.sv.gnu.org/emacs.git
		* IRC channel: irc.freenode.net/emacs


#####################################################################
Form edit

https://directory.fsf.org/wiki?title=Emacs&action=formedit&debug=true

if debian package == yes
	Package name: Readonly
	Version: Readonly
	Release date: Readonly
	Dependencies: Readonly
	Download link: Write
	VCS Checkout: Write
	IRC channel: Write
else
	Package name: Write
	Version: Write
	Release date: Write
	Dependencies: Write
	Download link: Write
	VCS Checkout: Write
	IRC channel: Write

Issues

Resource links

Bendikker has added hundreds (if not thousands) of resource links to various entries. I think we should import them if possible. Here's what he added to Emacs for example 

{{Resource
|Resource audience=Python (Ref)
|Resource URL=https://pypi.org/project/emacs
}}
{{Resource
|Resource audience=Debian (Ref) (R)
|Resource URL=https://tracker.debian.org/pkg/emacs23
}}
{{Resource
|Resource audience=Debian (Ref) (R)
|Resource URL=https://tracker.debian.org/pkg/emacs24
}}
{{Resource
|Resource audience=Debian (Ref) (R)
|Resource URL=https://tracker.debian.org/pkg/emacs25
}}
{{Resource
|Resource audience=Debian (Ref)
|Resource URL=https://tracker.debian.org/pkg/emacs
}}
{{Resource
|Resource audience=Savannah (Ref)
|Resource URL=https://savannah.gnu.org/projects/emacs
}}
{{Resource
|Resource kind=VCS Repository Webview
|Resource URL=https://git.savannah.gnu.org/cgit/emacs.git
}}
{{Resource
|Resource kind=Mailing List
|Resource URL=https://lists.gnu.org/mailman/listinfo/help-gnu-emacs
}}
{{Resource
|Resource kind=Mailing List
|Resource URL=https://lists.gnu.org/mailman/listinfo/bug-gnu-emacs
}}
{{Resource
|Resource kind=Mailing List
|Resource URL=https://lists.gnu.org/mailman/listinfo/emacs-devel
}}
{{Resource
|Resource kind=Mailing List
|Resource URL=https://lists.gnu.org/mailman/listinfo/gnu-system-discuss
}}
{{Resource
|Resource kind=Download
|Resource URL=https://ftp.gnu.org/gnu/emacs
}}

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the page “GNU Free Documentation License”.

The copyright and license notices on this page only apply to the text on this page. Any software or copyright-licenses or other similar notices described in this text has its own copyright notice and license, which can usually be found in the distribution or license text itself.

Retrieved from "https://directory.fsf.org/wiki?title=Free_Software_Directory:Infrastructure&oldid=84790"