Difference between revisions of "Free Software Directory:Backlog active"

From Free Software Directory
Jump to: navigation, search
(Question mark)
(Modify: Fixed IRC link bug.)
Line 365: Line 365:
 
=====Modify=====
 
=====Modify=====
  
* The "irc:" is displayed twice like this: irc://irc:is broken, see Template:Print entry. And if you remove irc://, it prepends http://. Form edit requires irc:: "must have the correct URL format, starting with "http""
 
 
* The "License" (friendly name: "License and version") input field is too liberal, it accepts any value and if the value is a non-existing page it will appear as a red link who will eventually be created by anyone who see it. Only licenses listed in https://www.gnu.org/licenses/license-list.html#SoftwareLicenses should be on our "License and version:" drop-down menu.
 
* The "License" (friendly name: "License and version") input field is too liberal, it accepts any value and if the value is a non-existing page it will appear as a red link who will eventually be created by anyone who see it. Only licenses listed in https://www.gnu.org/licenses/license-list.html#SoftwareLicenses should be on our "License and version:" drop-down menu.
 
** Possible solutions:
 
** Possible solutions:

Revision as of 14:20, 2 November 2018

Info

See our new infrastructure plan. Please report confidential or urgent issues to bug-directory@gnu.org.

Contents

SPDX-related issues

See the SPDX License List for license identifiers.

Software Package Data Exchange (SPDX) is a file format used to document information on the software licenses under which a given piece of computer software is distributed. The FSF has decided that we are going to switch to the SPDX identifiers (eg switch from MPL1.1 to MPL-1.1).

The issues described below must be fixed for the Free_Software_Directory:SPDX_Group to operate.

Related links:

General

  • Remove exposure of database strip marks from pages under License namespace. Currently some (if not most) licenses in License namespace have a "pre"/"nowiki" element in their call text to their associated template. This is problematic because the template itself already wraps the text in "pre", so the result is a exposure of the database strip marks. When in the "Read" tab, pages in License namespace that are affected by this issue have a yellow triangle next to the start of the license text, for example see https://directory.fsf.org/wiki?title=License:BitTorrentOSL1.1&oldid=72843
  • Make the license texts in License namespace preserve their initial space. Revisions 49613 and 49636 for the {{License}} template already implement a workaround for this issue, but this requires a mass replace in all calls to {{License}}, so as to replace |Full text= with |.
  • Make comments in License namespace point to license evaluation page of GNU project if they exist there. This should also add the benefit of having that page translated to the contributor or visitor/guest's language.
  • Autopopulate License verified date field if it is empty.
  • When people set the license for a project to be "GPL 2 or higher", it gets added to the autocompletion list for that field, even though that license isn't on the approved list at: Property:License. ("GPL 2 or higher" isn't on the list because it is misformatted. The correct entry is "GPLv2orlater".)

Replace License text in program entries

See Free_Software_Directory:SPDX_Group for text to replace

Avoid https://directory.fsf.org/wiki/Special:ReplaceText

  1. https://directory.fsf.org/wiki/Special:ReplaceText is limited to 250 pages per execution. That means to much job, "License:GPLv2orlater" is used in 4459 pages to give a picture.
  2. to slow, 10 pages modified in 3 minutes (https://directory.fsf.org/wiki/Special:RecentChanges monitored), or 250 pages in 90 minutes.
Original text:
|License=GPLv3orlater
Replacement text:
|License=GPL-3.0-or-later

Search in namespaces:Main
Replace only in category: Entry

Do not use regex like this, it will add "^|License=GPLv3orlater$" at the very beginning of the page:

Original text:
^|License=GPLv3orlater$
Replacement text:
|License=GPL-3.0-or-later

Use regular expressions
Search in namespaces:Main
Replace only in category: Entry

The license list should only be a drop down menu

Currently the license list found under "Licenses" in https://directory.fsf.org/wiki/Form:Entry is a hybrid of a dropdown but you can also add text by hand there too.

"input type=combobox" should not be changed to "input type=dropdown" in https://directory.fsf.org/wiki/Form:Entry because the edit form will not remember the licenses previously used, so saving a form without manually re-entering license may delete the license names from the entry by default.

Development testing

Tried to reset the license list on a dev server so we can rebuild it from scratch.

https://directory2p.fsf.org/wiki?title=Property:License&diff=prev&oldid=72725

has caused at lest 12135 jobs to be generated (maybe ca. 300 has been processed).

Cron will deal with it within 121 hours, or it is possible to run
"runjobs 10000" to run, say, 10000 jobs in one go (this does not
conflict with cron).

Basically it seems that we should be prepared to handle ca. 15000 jobs
quickly. Maybe the system can be tuned to do that in a reasonable timeframe.
It mostly depends on the MySQL database server.

One can check the amount of jobs left by running "wikijobs" (it is cached but for not very long).

Marcin

I deleted all licenses in https://directory2p.fsf.org/wiki/Property:License two days ago and there are no jobs left (https://directory2p.fsf.org/w/api.php?action=query&meta=siteinfo&siprop=statistics).

Some unexpected licenses, for example "BSD"

Question mark

Add {{#info:}}to https://directory.fsf.org/wiki/Form:Entry

'''License and version:'''{{#info:Press any key to quickly select entries that start with that character}}

Update the license field

Semantic MediaWiki issue:

I've updated https://directory.fsf.org/wiki/Property:License but https://directory.fsf.org/w/api.php?action=pfautocomplete&format=json&property=License&substr=MPL doesn't show the update. Same issue with the old server http://directory-dev.fsf.org/wiki/Property:License and http://directory-dev.fsf.org/w/api.php?action=pfautocomplete&format=json&property=License&substr=Foo

2018-08-08

(19:29:54) David_Hedlund: I've removed MPL1.1 (replaced by MPL-1.1), and MPLv2.0 (replaced by MPL-2.0) in https://directory.fsf.org/wiki/Property:License, but the "License and version:" drop down menu in https://directory.fsf.org/wiki?title=Sandbox&action=formedit&debug=true#tab=3__Licenses remain unchanged: Search for "MPL" in it and you'll find only "MPL1.1" and "MPLv2.0". I have clicked on More -> Refresh to purge the cache but it didn't help. Do you know how I can update the list?

Same with non-existing pages, example: https://directory.fsf.org/wiki?title=Sandbox6&action=formedit&debug=true#tab=3__Licenses

saper helped a lot on this issue.

Andrew said that it hopefully get fixed after they have solved `[directory] stale results from API call - https://rt.gnu.org/Ticket/Display.html?id=1314342`
(21:37:23) saper: David_Hedlund: https://directory.fsf.org/w/api.php?action=query&meta=siteinfo&siprop=statistics "jobs": 16181
(21:40:33) sudoman: i'm running them in batches of 200

2018-08-09
Hello David,

The job queue is currently at about 11 or 12 jobs, but it looks like the
changes for MPL licenses did not go through. i tried editing and saving
one of the MPL license pages and editing and saving template:license,
but this did not update those pages.

i'll look into this some more, but if you can find any hints about this,
that would be very helpful, but is not required. I suspect that it's an
issue with semantic mediawiki. perhaps it doesn't handle page renames
properly via the template:license code.

Thanks,
Andrew


(17:00:55) sudoman: David_Hedlund: i sent you an email about the directory.fsf.org job queue. it's currently at about 11 or 12 jobs, but it looks like the license names didn't get updated. i suspect that this is an issue with semantic mediawiki not properly handling page renames.
(18:37:01) David_Hedlund: sudoman: 1 job left!
(23:03:22) David_Hedlund: sudoman: "jobs": 877 - https://directory.fsf.org/w/api.php?action=query&meta=siteinfo&siprop=statistics -- I assume all old jobs are solved.
(23:07:05) David_Hedlund: sudoman: Is this ticket solved? [directory] stale results from API call - https://rt.gnu.org/Ticket/Display.html?id=1314342
(23:07:46) sudoman: David_Hedlund: it's not resolve yet...
(23:58:22) David_Hedlund: sudoman: Checking `'''License and version:''' {{{field|License|input type=combobox|values from property=License}}}` in https://directory.fsf.org/wiki/Form:Entry
(23:59:19) David_Hedlund: It's used by https://directory.fsf.org/wiki/Template:Project_license

After a full rebuild of SMW data with a command:

flock -w 1 /var/run/apache2/SMW.flock php /var/www/w/extensions/SemanticMediaWiki/maintenance/rebuildData.php -v  --report-runtime 

that took over 14 hours:

437206 IDs refreshed.

Memory used: 180260544 (b: 13003608, a: 193264152) with a runtime of 53171.62 sec (886.19 min)

https://directory2p.fsf.org/w/api.php?action=pfautocomplete&format=json&property=License&substr=MPL returns only "MPL", so looks like the old entries are gone.

Adding MPLx for testing
  • added MPLx
  • After that change there are 15459 jobs in the queue (previously zero).
  • Started flock -w 1 /var/run/apache2/SMW.flock php /var/www/w/extensions/SemanticMediaWiki/maintenance/rebuildData.php -v --report-runtime again. It seems there not so many stale data, so it seems to be faster.
MPLx assigned to Sandbox
  • Changed license to MPLx
  • 5 jobs generated in the queue:
2018-08-13T09:15:01-04:00 started 2018-08-13T09:15:01-04:00
2018-08-13 13:15:02 htmlCacheUpdate Sandbox table=templatelinks recursive=1 rootJobIsSelf=1 rootJobSignature=0f9daa65837ad83184ef098150a8a3801a659bd6 rootJobTimestamp=20180813131439 causeAction=page-edit causeAgent=unknown requestId=69dc4b69b6232fe75f17ee66 (id=5482276,timestamp=20180813131442) STARTING
2018-08-13 13:15:02 htmlCacheUpdate Sandbox table=templatelinks recursive=1 rootJobIsSelf=1 rootJobSignature=0f9daa65837ad83184ef098150a8a3801a659bd6 rootJobTimestamp=20180813131439 causeAction=page-edit causeAgent=unknown requestId=69dc4b69b6232fe75f17ee66 (id=5482276,timestamp=20180813131442) t=19 good
2018-08-13 13:15:02 enotifNotify Sandbox editor=Saper editorID=1696 timestamp=20180813131435 summary=license: MPLx for testing minorEdit=0 oldid=71795 watchers=[1501] pageStatus=changed requestId=69dc4b69b6232fe75f17ee66 (id=5482279,timestamp=20180813131443) STARTING
2018-08-13 13:15:02 enotifNotify Sandbox editor=Saper editorID=1696 timestamp=20180813131435 summary=license: MPLx for testing minorEdit=0 oldid=71795 watchers=[1501] pageStatus=changed requestId=69dc4b69b6232fe75f17ee66 (id=5482279,timestamp=20180813131443) t=15 good
2018-08-13 13:15:02 htmlCacheUpdate Sandbox table=redirect recursive=1 rootJobIsSelf=1 rootJobSignature=071d8bc99773d0c00cc4579b6b049e7c20840e82 rootJobTimestamp=20180813131439 causeAction=page-edit causeAgent=unknown requestId=69dc4b69b6232fe75f17ee66 (id=5482277,timestamp=20180813131442) STARTING
2018-08-13 13:15:02 htmlCacheUpdate Sandbox table=redirect recursive=1 rootJobIsSelf=1 rootJobSignature=071d8bc99773d0c00cc4579b6b049e7c20840e82 rootJobTimestamp=20180813131439 causeAction=page-edit causeAgent=unknown requestId=69dc4b69b6232fe75f17ee66 (id=5482277,timestamp=20180813131442) t=8 good
2018-08-13 13:15:02 recentChangesUpdate Special:RecentChanges type=cacheUpdate requestId=69dc4b69b6232fe75f17ee66 (id=5482278,timestamp=20180813131442) STARTING
2018-08-13 13:15:02 recentChangesUpdate Special:RecentChanges type=cacheUpdate requestId=69dc4b69b6232fe75f17ee66 (id=5482278,timestamp=20180813131442) t=51 good
2018-08-13 13:15:02 refreshLinksPrioritized Sandbox rootJobTimestamp=20180813131435 useRecursiveLinksUpdate=1 triggeringUser={"userId":1696,"userName":"Saper"} triggeringRevisionId=72711 causeAction=edit-page causeAgent=Saper requestId=69dc4b69b6232fe75f17ee66 (id=5482275,timestamp=20180813131438) STARTING
2018-08-13 13:15:03 refreshLinksPrioritized Sandbox rootJobTimestamp=20180813131435 useRecursiveLinksUpdate=1 triggeringUser={"userId":1696,"userName":"Saper"} triggeringRevisionId=72711 causeAction=edit-page causeAgent=Saper requestId=69dc4b69b6232fe75f17ee66 (id=5482275,timestamp=20180813131438) t=1200 good
2018-08-13T09:15:01-04:00 ended 2018-08-13T09:15:03-04:00

https://directory2p.fsf.org/w/api.php?action=pfautocomplete&format=json&property=License&substr=MPL gives

{"pfautocomplete":[{"title":"MPL"},{"title":"MPLx"}]}

General

  • There should be a space dedicated for scripts that can be downloaded by FSD volunteer. I suggest a link to a page in the side-bar that lists the scripts.
  • Proof read: I have emailed already emailed https://directory.fsf.org/wiki/Free_Software_Directory:Backlog#FSF_staff_tasks to Donald and asked him to suggest the sysadmins to have a look on this. However, we must proof-read the FSF staff tasks section so the FSF sysadmins clearly understand the text so they can start working on this.
  • Proposed features for the FSD
  • Category browsing
  • Fix search results, so that template data doesn't get dumped into the description below the page links. In other words, just put in the beginning of the description.
  • add a link for binary download?
  • need to implement mass-emails about email verification, and directions for changing preferences.
  • Add/update GNU packages
  • "Preview" button does not work for Template:Entry. It shows the unedited version, but I don' think there is a way around this from our end, as the displayed values are based on a query of the saved page.
  • Put a link to site dumps and info about the machine readability of that information on the main page.
  • Created HTML tooltips for the different fields in Form:Entry, or create a tutorial for Form:Entry that describes what each field is for.
  • Pages incorrectly created with Form:Bug_update (without the "Review:" namespace prefix) are queryable. If they are then moved to the Review: namespace, then they remain queryable. I'm not sure if a non-admin can edit the page, which could let them create a false entry in the directory.
  • Update the queries on GNU/Export to make sure the second column is the GNU package name property
  • Update each GNU project with the GNU package name
  • Add "table of contents" header above tabs inside noscript tags so that it only displays when javascript is turned off.
  • We need to let people submit bug reports, etc. without javascript enabled. If we could use something like a "noscript" tag, then we could display the following link: active&namespace=Review Example bug report link
  • 'new messages' text overlays the red FSF links bar, so you can't click it. Still, you can just click your 'talk' page link instead.
  • "New Packages RSS feed"'s license verification dates are messed up.
  • home page of every GNU package (yes, really) to be listed as <https://www.gnu.org/software/PKGNAME>
  • conditional statements should be created for various properties (e.g., "Audio:mp3" should result in "works-with::Audio" being set on a page automatically).
  • Implement an in-browser irc client that displays only when meetings are live on the Main Page in addition to the "The meeting is active right now!" (see Template:IRC_text).

Fixed

  • Fix search links: Fix links (multimedia, etc) at top of search results.

date command

Add: date --date='TZ="America/New_York" 12:00 this Fri'

Copyright current year

Add the current year in /var/www/w/skins/Vector/VectorTemplate.php "Copyright © 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017 Free Software Foundation, Inc."

Favicon

"I think I would just use the fsf.org (https://static.fsf.org/fsforg/img/favicon.ico) one for any fsf.org subdomain." - quidam

Special:Ask

Special:AdminLinks "Semantic search" is blank.

Comment: To fix all of the above MediaWiki debugging needs to be enabled. It is also good to see the PHP error log.  « Saper // @talk »  13:17, 27 January 2018 (EST)
It seems to work some how now asking for License:MPL  « Saper // @talk »  15:59, 8 August 2018 (EDT)

What links here?

We want the "What links here?" enabled in the sidebar, it's the standard procedure to maintain MediaWiki instances. It's not stored in https://directory.fsf.org/wiki?title=MediaWiki:Sidebar&action=edit. I got this response from #mediawiki "It's enabled by default on normal pages. I'd say there may be an extension installed on that wiki that's hiding it. Moonbook, https://directory.fsf.org/wiki/IceCat?useskin=monobook, has "What links here?". It's strange because it's not present in the original source code. it's not hidden with CSS or JavaScript"

Import

GNU packages

Make a script to generate XML files that syncs GNU with https://www.gnu.org/software/ which also list decommissioned GNU packages, a property that also should be imported, see https://directory.fsf.org/wiki/Property:Decommissioned/Obsolete

See https://directory.fsf.org/wiki/User_talk:Bendikker/cleanlinksgnusoftware for other examples.

Screenshots

https://screenshots.debian.net/

It's not possible to download the archive right now.

" The current version features a large JSON blog that gives you information about all packages and the related screenshots. You could load the screenshots directly from directory.fsf.org if you like. That's what several other websites like packages.debian.org do. The only challenge you would face might be to match the "upstream" package name to the software name you use in your database." -- Christoph Haas, the Debian Screenshots maintainer

Icons

The https://screenshots.debian.net/ maintainer said that he will add icons for Debian packages (provided by https://wiki.debian.org/AppStream) in the bug tracker: https://salsa.debian.org/debian/debshots/boards.

Make it possible to upload software icons in the edit form. Display the icons next to the title in the entries. I asked https://directory.fsf.org/wiki/User_talk:Vibhuti to do this (and to write down a howto in https://directory.fsf.org/wiki/Free_Software_Directory:Backlog_Admin_Group#How_to_add_file_upload_to_form_edit) since she made it possible to upload screenshots in the edit form recently. --David Hedlund (talk) 15:16, 20 April 2018 (EDT)

Other repo data

Wikidata

Take data from wikidata.org that's not in Debian's main repository.

XML update

Please study these XML outputs in IceCat

It's time to export the old add-on list https://directory.fsf.org/wiki/Free_Software_Directory:IceCat_extensions_(proposed)/ from the FSD, then import the new list with 600̈́+ add-ons, then merge the extra field from the backup to it.

My example (look for differences) when I want to import 600 add-ons:

{{Entry
|Name=NoScript
|Short description=Only enable JavaScript for sites you trust.
|Full description=The '''NoScript''' (aka '''NoScript Security Suite''') Firefox extension provides extra protection for Firefox, Seamonkey and other Mozilla-based browsers.  Its aim is to allow active content to run only from sites you trust, and protect yourself against XSS and Click-jacking attacks.  It is also useful to prevent Flash running on the browser.

But its main advantage could be the alerts given on its interface. There is an increasing number of Web sites that cannot run without non-free JavaScript programs downloaded and executed on the visitor's computer. Richard Stallman first raised this another concern with his article [https://www.gnu.org/philosophy/javascript-trap.html The JavaScript Trap], pointing out that most JavaScript programs are not freely licensed, and that even free software Web browsers are usually configured to download and run these non-free programs without informing the user.  So maybe NoScript could encourage people to keep informed about this related issue and [https://www.fsf.org/campaigns/freejs take action].

Anyway, NoScript is an extremely valuable Firefox extension for security. Its unique white-list based pre-emptive script blocking approach prevents exploitation of security vulnerabilities (known and even not known yet!) with no loss of functionality...
|Homepage URL=https://noscript.net/
|User level=intermediate
|Extension of=IceCat
|Documentation note=Install NoScript and enable ‘Forbid scripts globally’ to improve the security of your browser by preventing 0day JavaScript attacks. This is a drastic option as it will render many websites unusable as they rely heavily on JavaScript. NoScript offers a whitelist you can use to selectively enable JavaScript for sites you trust, but this is considered [https://www.torproject.org/docs/faq.html.en#TBBCanIBlockJS especially bad for your anonymity] if you're using NoScript with Tor.
|Related projects=GNU LibreJS, NoScript Anywhere
|Version identifier=2.6.9.25.1-signed
|Version date=2015/05/23
|Version status=stable
|Version download=https://addons.mozilla.org/firefox/downloads/file/318124/noscript_security_suite-2.6.9.25-sm+fn+fx.xpi
|Last review by=David Hedlund
|Last review date=2016/04/21
|Status=
|Is GNU=No
}}
{{Project license
|License=GPLv2
|License verified by=enyst
|License verified date=2013/03/24
}}
{{Resource
|Resource audience=Users
|Resource kind=Forum
|Resource URL=https://forums.informaction.com/viewforum.php?f=3
}}
{{Software category
|Internet-application=privacy
|Use=internet-application, security
}}
{{Featured}}

Consistent CSS styling

Some custom templates have CSS style="" clauses scattered in them. One of such clauses was responsible for too narrow columns in the description. I'd suggest to:

  1. Check all local style clauses in our custom code
  2. Determine if they can be more generic. For example, colors, paddings, line widths should be unified to achieve visual consistency.
  3. A set of CSS classes should be designed to replace style clauses
  4. CSS class naming scheme should be decided
  5. New CSS classes should be introduced to MediaWiki:Common.css
  6. style clauses should be replaced with their class="css-class" counterparts.

Backlog Admin Group tasks

These are issues that has effectively been collaborated in the Backlog Admin Group.

IRC bot

Semantic Forms

Property:Resource URL is of type string/text, this allows any value, even phone numbers, SIP, XMPP, etc. Perhaps we should use something along the lines of "Allows pattern" special property in order to avoid abuses. Property:Resource URL is the one currently used in the entry form when someone fills the "resource URL".

Add

To "Edit with form" (Example: https://directory.fsf.org/wiki?title=IceCat&action=formedit&debug=true#tab=1__General_info):

Modify
  • The "License" (friendly name: "License and version") input field is too liberal, it accepts any value and if the value is a non-existing page it will appear as a red link who will eventually be created by anyone who see it. Only licenses listed in https://www.gnu.org/licenses/license-list.html#SoftwareLicenses should be on our "License and version:" drop-down menu.
    • Possible solutions:
      1. Change type from combobox to dropdown. Attempted, but reverted due to issues.
        • This would make the form edit misbehave, as it would not find the already existing values in old entries that are set before the change. Thus, in the form edit, if an already existing entry/page has MPLv2.0 set as License parameter for {{Project license}}, then it won't find it since it expects License:MPLv2.0. Making {{Project license}} set this doesn't work (it was already tried), the only fix for the already existing entries seems to be a mass-replace, so that every call for {{Project license}} that is like this {{Project license |License=MPLv2.0 }} becomes like this {{Project license |License=License:MPLv2.0 }}. The bug described in this item doesn't affect the show/read tabs of each entry.
          • Before attempting to fix this, see if fixing the item below would also fix this one.
        • Must make sure that the allowed values for Property:License or for that license-related input only point to pages at the License namespace. For example, currently Property:License describes Boost as one of the allowed values, however it seems to refer to Boost, not License:Boost.

Future

proprietary program search

When a person searches for a proprietary program, a list of free software replacements for that program should appear in the search results, *however*, the proprietary program and its name should not appear anywhere on the wiki. To achieve this functionality, we may want to use the Enhanced Retrieval extension (obsolete).

License verification

Licenses and license notifications should preferably be verified by a license compliance software system (like FOSSology).

When doing license verification by hand, we need to ensure two things:

  1. More than one person checks a license
  2. We have a system in place for training volunteers (via some sort of mentorship) on how to properly check a project's license.

Point system and mentoring

Some ideas we have discussed is that we might have some sort of scoring system, so that more experienced individuals who check a license have more "points" than people less experienced. Let's say we have three tiers: level-1, level-2, level-3; with level-1 being the most experienced. A check by a level-1 person might be worth 15 points, a level-2 might be worth 10 points, and a check by a level-3 person might be worth 5 points. It might be that a license is properly "checked" if it has a score of at least 25 points. To coordinate this work we might consider using an extension such as Semantic Tasks.

But, more important than the point system is ensuring that level-3 individuals are not only verified by level-1 individuals, but, that some 1-on-1 guidance and help/mentoring is provided to level-3 individuals from level-1 individuals.

FSF staff tasks

Please give us a dedicated bug tracker. There are to many bugs and issues on this page now to work with it effectively. Everything is a mess.

Some work, like modifying the source files of MediaWiki, cannot be done by MediaWiki users themselves. We gather such tasks here so the FSF staff can help us fix them.

A dedicated bug tracker for the Directory

This page has to many bugs to effectively organize them. We need a dedicated bug tracker for the Directory.

The concept for public bug trackers for FSF run websites has been posted in Directory-Discuss mailing-list.

Matt Lee said that he back in January 2017 but nothing happened and he never replied!

Ian discussed this with Ruben who suggested GitLab Community Edition since ruben has used it for the trisquel.info. Ian is willing to adapt it but found [some problems with it regarding the license fields]. Ian plans to submit a patch programmed in Ruby to solve the issue.

Internationalization and localization

https://en.wikipedia.org/wiki/Free_Software_Directory is available in these languages at this writing:

Languages:
Español
Français
Italiano
Nederlands
日本語
Русский
中文

We need to enable internationalization and localization in our MediaWiki instance so volunteers easily can add new translated pages. For example Collection:IOS (4 languages) and Collection:Windows (3 languages) need the translation links in the sidebar.

Monitoring MediaWiki

Job queue length

alias wikijobs='curl -s '\''https://directory2p.fsf.org/w/api.php?action=query&meta=siteinfo&siprop=statistics&format=json'\'' | jq -r .query.statistics.jobs'

Full rebuild of Semantic Mediawiki data took over 14 hours, after that there were 0 jobs in the queue

437206 IDs refreshed.

Memory used: 180260544 (b: 13003608, a: 193264152) with a runtime of 53171.62 sec (886.19 min)

Semantic MediaWiki

Licenses

  • List incomplete IceCat add-ons. Free Software Directory:IceCat extensions (proposed)/FSD bugs issue 1
  • Require license
    • Entries with empty Licence fields should not be able to approve. It should only be possible to approve entries with filled License fields. This policy must include the automatic approval when pages are save.
    • Add required text for License. For Create, when a user try to save the page with Homepage not filled out the text There were errors with your form input; see below. will appear on the Save tab. On the General info tab the text Homepage URL (required) cannot be blank text will appear. However, there are no text that says that License fields must be filled if they are not.
  • List unapproved IceCat add-ons. We could reuse the source of https://directory.fsf.org/wiki?title=Special:ApprovedRevs&show=unapproved so we can tweak to show only unapproved IceCat add-ons.
  • License list is incomplete. How do we list all 251 licenses found in https://directory.fsf.org/wiki/Category:License to "Edit -> Licenses" for entries?
  • Sync license data:
(18:11:38) David_Hedlund: saper: It would be super nice if we could generate https://directory.fsf.org/wiki/Property:License from https://directory.fsf.org/wiki?title=Special%3APrefixIndex&prefix=license%3A&namespace=0 -- do you know if that's possible?
(18:29:24) saper: David_Hedlund: see https://www.semantic-mediawiki.org/wiki/Help:Special_property_Allows_value https://www.semantic-mediawiki.org/wiki/Help:Special_property_Allows_value_list https://www.semantic-mediawiki.org/wiki/Help:Special_property_Allows_pattern. I don't think it is possible (except for a bot rewriting it). The reason is that querying lists of pages on every page load will be slow and hard to cache. 
(18:29:30) saper: (but alas Semantic MediaWiki is already slow).
(18:31:40) saper: or compare things and spit out the differences

Link text

IRC (temporarily fixed)

The IRC resource is displayed twice like this on IceCat for example.

IRC general channel
    [irc:chat.freenode.net/icecat irc:chat.freenode.net/icecat] 
  • I attempted to fix this but it seems that something is interfering with the formatting. Strangely enough, if I make a simple page without the form, and use the IRC template, it works as expected. I don't know if this is the case, but I think there is some template interfering.--Adfeno (talk) 23:23, 26 January 2018 (EST)
  • As requested in #fsf IRC channel, here is the comment I made there, describing my studies so far:
    • It basically involves three templates (it might involve more, but I only researched three so far) {{Print entry}} {{Entry}} and {{Show}}. There is a fourth one ({{IRC}}) but I think we might be able to replace it ({{Print entry}} calls it, so instead of calling {{IRC}}: we can take the code from it and insert in the caller template).
      • I think that #replace will just mask the underlying issue. --David Hedlund (talk) 10:49, 27 January 2018 (EST)
    • I think there is so far the following possibilities:
      1. an issue between the templates used;
      2. an issue with the #ask: queries and the template result format;
      3. we have reached a setting limit that makes the formatting not work (perhaps a limit on the template result format, although I have tried to remove the call to {{IRC}} but it didn't work, we might have to make tests by simplifying the template result calls even further to see if this is indeed the problematic setting);
      4. an issue with the "IRC [level name]" properties when setting (#set:) or their query and display.
    I think we can make use of #nowiki or #- display parameters when querying for the "IRC [level" properties of an entry (this is queried for in {{Show}}) in order to take the property value in raw (example: irc:chat.freenode.net/fsf) but then we will need to change either {{Print entry}} or {{IRC}} (the first one is the problematic, so I recommend to test with it first) so that it removes irc: but also allows any other : (because the URL scheme allows port numbers).
    --Adfeno (talk) 08:47, 27 January 2018 (EST)
  • While MediaWiki understands IRC URLs very well, Our copy Semantic MediaWiki gets confused if a property of type URL has a protocol different from HTTP, this problem also happens if the property definition/setting is invisible (using #set:), but in all cases it works as expected if the protocol is HTTP. I don't know if this is a bug in Semantic MediaWiki (or MediaWiki) but this affects at least our copy of it. An example of the issue with IRC property can be seen in: https://directory.fsf.org/wiki?title=Free_Software_Directory:Sandbox&oldid=48884 --Adfeno (talk) 11:32, 27 January 2018 (EST)
  • As of SMW 2.5.1, http https and ftp URL schemes will be printed with double slashes, all other without: https://github.com/SemanticMediaWiki/SemanticMediaWiki/blob/568d058ea68859d134654a959001ba0739c781e3/includes/dataitems/SMW_DI_URI.php#L68-L70  « Saper // @talk »  12:49, 27 January 2018 (EST)
Log
Multiple software categories (temporarily fixed)

Not only are the text merged but also the URL in the link. For IceCat: <a href="/wiki?title=Category/Programming-language/C_C%2B%2B_javascript_rust&action=edit&redlink=1" class="new" title="Category/Programming-language/C C++ javascript rust (page does not exist)">Programming-language:C C++ javascript rust</a>

Log

Remove duplication

Remove duplicated entries

The value of |Extension of= is displayed twice in some entries (one year ago perhaps 5 duplicated were displayed but John Sullivan and/or Joshua Gay managed to fix most of the duplicates either by himself or someone else). For example, "Add to Search Bar" appear twice there, but "Aardvark" appear only once.

The duplicates also affect lists that the entries are in, see this page for test purpose that I've set up: https://directory.fsf.org/wiki/IceCat

       > I contacted Joshua in March this year about this and he responded.

       > "I will look into this. I am not sure why the redirects are making it
       > show-up twice. It shouldn't. However fixing it is not a simple matter of
       > removing them -- it is a bug in the software."

    Please report the bug to sysadmin@gnu.org.

    --
    Dr Richard Stallman
    President, Free Software Foundation (gnu.org, fsf.org)
    Internet Hall-of-Famer (internethalloffame.org)
    Skype: No way! See stallman.org/skype.html.

Remove duplicated short description

sysadmin@gnu.org ticket: [gnu.org #1205033]

Short description now appear automatically on software entry pages.

We had to copy the short description into the full description before thus creating duplicated text. The job to do now is to remove those duplicates. I added this to the Backlog.

Faceted search

Make an advanced search page used to filter out entries based on their SMW data.

Examples of filters:

  • Is GNU
  • Accepts cryptocurrency donation
  • User level
  • Categories...

MediaWiki

Special pages

Broken pages in https://directory.fsf.org/wiki/Special:SpecialPages :

Maintenance reports

  • Broken redirects
  • Dead-end pages
  • Double redirects
  • Oldest pages
  • Orphaned pages
  • Pages with the fewest revisions
  • Pages without language links
  • Protected titles -- doesn't work despite that Protected pages list pages
  • Uncategorized categories
  • Uncategorized files
  • Uncategorized pages
  • Uncategorized templates
  • Unused categories
  • Unused files
  • Unused templates
  • Unwatched pages
  • Wanted categories
  • Wanted files
  • Wanted pages
  • Wanted templates

Lists of pages

  • List of redirects

Media reports and uploads

  • List of files with duplicates
  • MIME search -- "This special page is disabled for performance reasons."
  • Media statistics

High use pages

  • Most linked-to files
  • Most linked-to pages
  • Most transcluded pages
  • Pages with the most categories
  • Pages with the most interwikis
  • Pages with the most revisions

Page tools

Extension:Approved Revs

  • Some pages show up in https://directory.fsf.org/wiki?title=Special:ApprovedRevs&limit=500&offset=0&show=invalid. It's not possible to approve or unapprove the Collection namespace. I was told it could be solved by playing with $egApprovedRevsNamespaces in LocalSettings.php.
  • Approved pages (and their redirects) becomes unapproved when they are Moved.
  • On the entry pages, there is a bug where if a page has had at least one approval and a non-admin edits it, the entry page shows the latest edit instead of the last approved revision.

Extension:UserFunctions

All the bureaucrats represents their user access level on their user page with static text. Example for User:Donaldr3:

My name is Donald Robertson and I am an administrator and bureaucrat of the Free Software Directory.

The problem with that method is that each time the user access level changes the text has should be updated, for example when someone from the FSF gets retired. But this typically doesn't happen because it's inconvenient.


With https://www.mediawiki.org/wiki/Extension:UserFunctions we can use this on the user pages for administrators and bureaucrats so they can represent their user access level on their user page dynamically. We can add this to a template that we add to the top of the users pages for example:

My name is {{#realname:alt}} and I am an {{#ifingroup:bureaucrat| [https://directory.fsf.org/wiki?title=Special%3AListUsers&username={{#username:alt}}&group=bureaucrat&limit=1 administrator and bureaucrat] |}} {{#ifingroup:administrator| [https://directory.fsf.org/wiki?title=Special%3AListUsers&username={{#username:alt}}&group=bureaucrat&limit=1 administrator] |}} of the Free Software Directory.

Performance tuning

https://www.mediawiki.org/wiki/Manual:Performance_tuning

A lot of contributors have complained that the FSD has been slow in the past years. The most damaging aspect with this is that we are loosing contributors. It must with other words be fixed if we want more contributors -- something that has been suggested as being one of the most important thing with the project at all.

  • Ask FSF admins to provide an exact copy of the environment (database + software) so that can be reproduced locally and properly troubleshoot.
  • Switching to PHP 7 might help a bit but I think we have a problem with writing data to Semantic MediaWiki, this might me an issue with forms.
  • Install memcached, or find some other way to improve performance of the site. We already have Zend OPcache, for opcache, and APCu, for data store, installed and enabled. Memcached is like APCu, but slower.
Session data

Disclaimer: Andrew was said that this is most likely not an issue with the programs that hosts the Directory. -- 2018-05-17

It may take a minute to move a page. Also, I get this message after ~1 minute (should be at least 5 minutes) of inactive form editing:

Sorry! We could not process your edit due to a loss of session data.

You might have been logged out. Please verify that you're still logged in and try again. If it still does not work, try logging out and logging back in, and check that your browser allows cookies from this site.

One must note that this expiration is common practice in order to avoid fake authentication attacks. We must instead take the message at face value and think: "What can cause someone to stay various minutes editing a page? And how can we segment the edit so that this doesn't happen?"

Steps to reproduce:

  1. Click on the edit link of a page (or even the "edit with form" link).
  2. Do some edit but don't submit.
  3. Wait at least 10min without visiting nor browsing any page related to both the FSD and the FSF (because these are linked by a CAS, central authentication server/service).
  4. Submit the edit.
    • The message appears. From this point on, if you try to submit again, the data is sent as expected.

Import templates

Templates to display "EST" or "EDT"

Workaround

No template works if copy/pasted directly from Wikipedia, not even those which print the current time zone offset in North America, either they require a setting to be passed, or they simply fail to find some string. Fortunately we managed to make Template:TZAbbrev which also has a default value that allows easily spotting for problems, besides accepting an optional parameter for testing, the documentation is in the template's page.

Misc

wgAllowUserJs

Personal JavaScripts are only available if $wgAllowUserJs is set to true. - https://www.mediawiki.org/wiki/Manual:Interface/JavaScript#Personal_scripts. wgAllowUserJs are set to true at wikipedia.org but not at directory.fsf.org (see policy).

I cannot use the tool mw.loader.load('//meta.wikimedia.org/w/index.php?title=User:Hedonil/XTools/XTools.js&action=raw&ctype=text/javascript'); in my User:David_Hedlund/common.js.
Hotcat is also pretty interesting: This code is MW version safe. It should run on any MediaWiki installation >= MW 1.15. Note that HotCat is supposed to run with or without jQuery, and also on older installations that do not yet have window.mw. - https://commons.wikimedia.org/wiki/MediaWiki:Gadget-HotCat.js

Change unapproved revs to category

Right now we have an issue where packages that aren't quite ready to be added to the Directory languish in unapproved revs. The packages don't need review; they need ongoing help to fix freedom issues. We want to keep these pages not listed in the Directory, but move them to like an Archive listing, where people can continue to file bugs and work with maintainers to fix the problems. But we want to get them out of unapprovedrevs, which is our listing for newly added packages that haven't been approved. mutante, a wikimedia amin, suggested that we just use categories rather than a special page. We can set the category automatically when people create the page, but then will later be able to change the category to either approved or archive as needed. This would also make it easier to take something down without deleting it, if freedom issues arise.

Suggested solution: I think SMW can evaluate unapproved/uncategorized entries. However, https://directory.fsf.org/wiki/Special:UncategorizedPages (and other #Special pages) are empty. --David Hedlund (talk) 15:39, 16 March 2018 (EDT)

Add a new name space: Group

I suggest a new name space "Group".

W3M / text based browser is hard to use forms

report from mailing list:

I just managed to submit an entry for my "dtas" software, but it was a bit painful since https://directory.fsf.org/wiki/Form:Entry leads to a "loading..." page which requires JavaScript.

He noticed that the javascript was trying to load a specific page, so he loaded it himself (directory.fsf.org/wiki/Special:FormEdit/Entry/dtas) and was able to edit the form.

Category:Candidates for deletion

Donald pointed out that https://directory.fsf.org/wiki/Category:Candidates_for_deletion lists pages that don't include the Category:Candidates_for_deletion tag:

Category/Business/pim
Category/Editor
Category/Localization
Category/Localization/language-adaptations
Category/Text-creation
Category/Use/organizing
Category/Works-with/pim
Category/Works-with/text

Suggestions

Upgrade history

It seems unprobable that the Semantic MediaWiki issues appeared on their own, most probably MediaWiki extensions got updated and some things do no longer work as before. Would be good to know the history of upgrades to identify changes that has been made and how they have affected this wiki.  « Saper // @talk »  10:58, 27 January 2018 (EST)

Donald

Logos

Meetings

IceCat

I'd like to have a IRC meetings with Ruben at least a few times per year. Suggested plan: 1 hour every 3rd month.

Backlog Admin Group (BAG) and FSF Tech Team

I suggest a voice conference between the BAG and the FSF. Suggested plan: 1 hour every 3rd month.

Mozilla / FSD joint project

Mozilla closes to many relevant issues:

Solution: I think the FSF should help the Project Team for IceCat WebExtension to have IRC meetings with Mozilla. Suggested plan: 1 hour every 3rd month.

Blog template

Disclaimer: This has been implemented since "No Friday Free Software Directory IRC meetup on Friday July 20th" in https://www.fsf.org/blogs/directory. Check a few more blogs to make sure it's well implemented.

Sent to donald@fsf.org

What do you think about this?: Move the paragraph with unique text to
the top of the body text and add the remaining text under a section
called "About the Free Software Directory". Example for
https://www.fsf.org/blogs/directory/friday-free-software-directory-irc-meetup-time-june-22nd-starting-at-12-00-p-m-edt-16-00-utc-1

##############################

Current version

Join the FSF and friends Friday, June 22nd, from 12:00 p.m. to 3 p.m.
EDT (16:00 to 19:00 UTC) to help improve the Free Software Directory
this week by helping to promote it.

Help improve the Free Software Directory by adding new entries and
updating existing ones. Every Friday we meet on IRC in the #fsf channel
on irc.freenode.org.

Tens of thousands of people visit directory.fsf.org each month to
discover free software. Each entry in the Directory contains a wealth of
useful information, from basic category and descriptions, to providing
detailed info about version control, IRC channels, documentation, and
licensing info that has been carefully checked by FSF staff and trained
volunteers.

When a user comes to the Directory, they know that everything in it is
free software, has only free dependencies, and runs on a free OS. With
over 16,000 entries, it is a massive repository of information about
free software.

While the Directory has been and continues to be a great resource to the
world for many years now, it has the potential to be a resource of even
greater value. But it needs your help! And since it's a MediaWiki
instance, it's easy for anyone to edit and contribute to the Directory.

A few weeks back we launched a new Project Team to create a logo that
packages can use to show off that they are included in the Directory.
But at this time, we still haven't found a team captain for the project.
This project could really help promote the Directory, as well as helping
packages to demonstrate that they care about software freedom. This
week, we'll be working to recruit a team captain, as well as trying to
get the project off the ground.

If you are eager to help, and you can't wait or are simply unable to
make it onto IRC on Friday, our participation guide will provide you
with all the information you need to get started on helping the
Directory today! There are also weekly Directory Meeting pages that
everyone is welcome to contribute to before, during, and after each
meeting. To see the meeting start time in your time zone, run this in
GNU bash: date --date='TZ="America/New_York" 12:00 this Fri'


##############################

Proposed template.

Join the FSF and friends Friday, June 22nd, from 12:00 p.m. to 3 p.m.
EDT (16:00 to 19:00 UTC) to help improve the Free Software Directory
this week by helping to promote it.

A few weeks back we launched a new Project Team to create a logo that
packages can use to show off that they are included in the Directory.
But at this time, we still haven't found a team captain for the project.
This project could really help promote the Directory, as well as helping
packages to demonstrate that they care about software freedom. This
week, we'll be working to recruit a team captain, as well as trying to
get the project off the ground.

[About the Free Software Directory]

Help improve the Free Software Directory by adding new entries and
updating existing ones. Every Friday we meet on IRC in the #fsf channel
on irc.freenode.org.

Tens of thousands of people visit directory.fsf.org each month to
discover free software. Each entry in the Directory contains a wealth of
useful information, from basic category and descriptions, to providing
detailed info about version control, IRC channels, documentation, and
licensing info that has been carefully checked by FSF staff and trained
volunteers.

When a user comes to the Directory, they know that everything in it is
free software, has only free dependencies, and runs on a free OS. With
over 16,000 entries, it is a massive repository of information about
free software.

While the Directory has been and continues to be a great resource to the
world for many years now, it has the potential to be a resource of even
greater value. But it needs your help! And since it's a MediaWiki
instance, it's easy for anyone to edit and contribute to the Directory.

If you are eager to help, and you can't wait or are simply unable to
make it onto IRC on Friday, our participation guide will provide you
with all the information you need to get started on helping the
Directory today! There are also weekly Directory Meeting pages that
everyone is welcome to contribute to before, during, and after each
meeting. To see the meeting start time in your time zone, run this in
GNU bash: date --date='TZ="America/New_York" 12:00 this Fri'

GPL

https://github.com/mozilla/addons-server/issues/8707 -- Mozilla (and perhaps most other developers) don't avoid the term "GNU General Public License, version 2.0" (or "GNU General Public License, version 3.0") as Ian pointed out later in that issue (https://github.com/mozilla/addons-server/issues/8707#issuecomment-401471396). Is the plan to split the license into two different licenses to avoid this issue completely: "GNU General Public License, version 4.0 only", "GNU General Public License, version 4.0 or later"?

Donald: "only" and "or later" are handled in license notices

New Antifeature

I want to add "officeonlinesystems PDF editor online" (https://addons.mozilla.org/en-US/firefox/addon/editor-pdf-online/) to the Antifeature as an example for one of the following

please read about the program and tell me which section that match best.

Software issues

Server-side

The issues here are note issues in the Directory but in programs used by the project.

CASAuth

MediaWiki

Revision entry on plain oldiff page displays "$7"

The bug can be seen in the revision summary next to the page title. Go to the history of a page, and click on any of the links that are in the dates (example "15:14, 27 January 2018‎").

This applies to the

  • main namespace, example [1]: Revision as of 17:08, 25 November 2016 by David Hedlund (talk | contribs | block)$7
  • project's namespace, example [2]: Revision as of 07:36, 24 October 2016 by David Hedlund (talk | contribs | block)$7
  • template namespace, example [3]: Revision as of 16:09, 27 January 2018 by David Hedlund (talk | contribs | block)$7

It doesn't affect:

  • property namespace (example: [4])
  • category namespace (example: [5])
  • Software software pages do not display revision summary at all so they are not either affected (example: [6]).

The $7 comes from the revision-info message (which can be seen at MediaWiki:Revision-info).

Client-side

The programs listed here has bugs that affect compatibility when you browse this website:

JavaScript errors

InvalidAccessError: A parameter or an operation is not supported by the underlying object ext.headertabs.core.js:41
TypeError: ss.rules is undefined[Learn More] wiki:477:35

might be mediawiki issue

TypeError: pf.select2.base is not a constructor[Learn More] ext.pf.select2.combobox.js:35:23
TypeError: pf.select2.base is not a constructor[Learn More] ext.pf.select2.tokens.js:36:21
initFunction is not a function $.fn.PageForms_registerInputInit
jQuery.Deferred exception: initFunction is not a function $.fn.PageForms_registerInputInit/<@https://directory.fsf.org/w/extensions/PageForms/libs/PageForms.js?0b6cb:330:17
resolve/</mightThrow@https://directory.fsf.org/w/load.php?debug=true&lang=en&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0hndq7l:3583:21
resolve/</process<@https://directory.fsf.org/w/load.php?debug=true&lang=en&modules=jquery%2Cmediawiki&only=scripts&skin=vector&version=0hndq7l:3651:12

URL: https://directory.fsf.org/wiki?title=Sandbox&action=formedit&debug=true#tab=3__Licenses

Related to date picker initialization in input_23

Outreachy Tasks

These are some of the tasks that will be worked on by Outreachy interns during summer 2018

  • Improve responsiveness for mobile devices
  • Sync Debian package info into the directory

Logos

Donald suggested logos in different sizes with HTML caption "Listed in the Free Software Directory". "something that packages could put on their website to show that they are free software and included in the directory". See these logos for example:

"The Free Software Directory (FSD, or simply Directory) is a project of..." - https://directory.fsf.org/wiki/Main_Page

Proposed text on the logo



Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the page “GNU Free Documentation License”.

The copyright and license notices on this page only apply to the text on this page. Any software or copyright-licenses or other similar notices described in this text has its own copyright notice and license, which can usually be found in the distribution or license text itself.