Free Software Directory:Participate/Script aid

From Free Software Directory
Jump to: navigation, search


This script aims to help contributors on finding things that might be needed to consider during evaluation of the project/software.

Besides searching for common licensing terms, it lists all the MIME/media types of files in the current directory, so that one can find cases of files without complete and corresponding source and also find JavaScript files to look or insert GNU LibreJS syntax.

To tackle cases of files which are partly JavaScript, it also scans for words related to JS and event handlers.

As an extra precaution, an extra pass for other possible problematic words is made.

There is also a field for notes which is always left empty for the evaluator to do the appropriate observations or even insert marks of continuation for future resume of the review.

The output is in CSV format, making it suitable for parsing by other software, such as LibreOffice Calc, GNU R and even GNU Awk. For the last case, before doing so, set FPAT to ([^,]*)|(\"[^\"]+\") and RS to \r\n, This variable setting was based on the related section in the GNU Awk User's Guide and on Awk's Texinfo/Info page.

In all cases, when importing, make sure that the selected language is English, so that "TRUE" and "FALSE" can be translated as correct boolean representation in your language of choice. Also, be aware of false-positives.

You're welcome to contribute to this script and add your name and contact information to the copyright notice of the script.


First, it's recommended to grant executable permissisns for your user, like so:

chmod u+x [path to FSD Script]

Then, take the complete corresponding source of the project being evaluated (e.g.: when using git clone, you can accomplish this using the --recursive option.

git clone --recursive [Some git repository.]
cd [Directory created by git]
[Script aid.] > [Desired text file to store output.]; printf '\a'

printf '\a' can be replaced by a command to play an audio file of your choice.

Now leave the script to do its work and wait for the sound clue to continue working on the evaluation.

After this, you might be wondering: how to go exactly to the points where the script found something, on each file? For this purpose there is the --print-ERE (-p) option. It prints an extended regular expression (ERE) that can be reused by you in any text editor to look for the same things, per file.

For example, you can do this:

less -p "$([Script aid.] -p)" "file1" "file2" "fileN"



# FSD Script Helps user evaluate entries for the Free Software Directory.
# Copyright (C) 2016, 2018, 2020, 2021  Adonay "adfeno" Felipe Nogueira <> <>

# This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

# This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU Affero General Public License for more details.

# You should have received a copy of the GNU Affero General Public License along with this program.  If not, see <>.

# For more information on the purpose and caveats of this script, see <>.

# # Dependencies

# * GNU bash;
# * any implementation of the following POSIX utilities:
#	* file, whose -i option prints MIME type and character set;
#	* find;
#	* grep;
#	* printf;
#	* sed;
#	* tr.

# # Usage

# It's recommended to grant executable permisisons for your user, like so:

# chmod u+x [path to FSD Script]

# Then, change to a directory containing the source files.

# Now do:

# [path to FSD Script] [options] > [path to the output .CSV file]

# Some options prevent normal output, so you could just do:

# [path to FSD Script] [options]

# # Options

# ## '--print-ERE' ('-p')

# Prevent normal output, and print an extended regular expression (ERE) that can be reused for checks against any file.

# This can be reused against each file with a text editor's regular expression search.

# For example, with GNU less, one can do:

# less -p "$([path to FSD Script] -p)" "file1" "file2" "fileN"

# … and then press N to go to each next match in the first file. Shift + N searches backwards in the same file.

# # Extended regular expressions (ERE) to match possible licensing issues

# Copyright symbol

# Use GNU bash's += assignment to append to an existing variable.
# Agreement
# Allowed
# As is
# Condition
# Copyright, copyleft, copyfarleft, copyfair, copymi
# EULA, exclusive
# Forbid
# License abbreviations
# Law
# Liable
# Responsible
# License
# Notice
# Patent, right
# Terms
# Trade
# Transfer, warrant

# # Extended regular expression (ERE) to match JavaScript issues

# Start of boundary

# Script tag or addEventListener() function

# Start of after, before, on variations

# Abort, autocomplete, blur, cancel, canplay
# ( Cue, duration, hash, language, rate, readystate, volume ) … change
# Click, close, contextmenu
# Drag and drop
# Emptied, ended, error, focus, input, invalid
# Key presses
# Load
# Start, message
# Mouse
# Connectivity and page
# Play, popstate, print, progress
# Reset, resize, scroll, seek
# Select, show, sort, stalled, storage, submit
# Suspend, timeupdate, toggle, waiting

# End of after, before, on variations

# End of boundary

# # Extended regular expression (ERE) to match possible other issues

# Google
# Microsoft, Facebook, WhatsApp, Telegram
# CDNs, Amazon, CloudFlare
# Apple, Uber, AirBnB
# System distributions
# DRM, Chrome and derivated

if [[ $1 == "--print-ERE" ||
	$1 == "-p" ]]; then

	# Print the extended regular expressions so that you can use then later.

separate_mime_from_charset() {
	# * Arguments
	#	* $1: file path to inspect MIME type and character set.
	# * Standard output
	#	* Comma-separated pair consisting of MIME type and character set.

	# Make use of -i to get MIME type and character set.
	# sed does the split and tr deletes line feeds/new lines.
	file -hi "$1" \
		| sed \
				s/: /",/g
				s/; charset=/,/g
			}' \
				| tr -d "\n"

was_ere_found() {
	# * Arguments
	#	* $1: extended regular expression to be searched;
	#	* $2: file path to look for given extended regular expression.
	# * Standard output
	#	* If a matche was found
	#		* String "TRUE"
	#	* Else
	#		* String "FALSE"

	grep -Eiq "$1" "$2"
	[ $? -eq 0 ] && printf "TRUE" || printf "FALSE"

# Use GNU bash's -f option to export functions for use with find.
export -f separate_mime_from_charset
export -f was_ere_found

printf '"Path","MIME","Charset","Licensing","JavaScript","Other","Notes"\r\n'
find "." \
	\( \
		-type d \
		\( -name '.cvs' -o -name '.hg' -o -name '.git' -o -name '.svn' \) \
	\) -prune -o \
	! -type d \
	-exec bash -c \
		'separate_mime_from_charset "{}"' \; \
	-exec printf ',' \; \
	-exec bash -c \
		'was_ere_found "'"$FSD_SCRIPT_AID_LICENSING_ERE"'" "{}"' \; \
	-exec printf ',' \; \
	-exec bash -c \
		'was_ere_found "'"$FSD_SCRIPT_AID_JAVASCRIPT_ERE"'" "{}"' \; \
	-exec printf ',' \; \
	-exec bash -c \
		'was_ere_found "'"$FSD_SCRIPT_AID_OTHER_ERE"'" "{}"' \; \
	-exec printf ',' \; \
	-exec printf '\r\n' \;

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the page “GNU Free Documentation License”.

The copyright and license notices on this page only apply to the text on this page. Any software or copyright-licenses or other similar notices described in this text has its own copyright notice and license, which can usually be found in the distribution or license text itself.