Difference between revisions of "Free Software Directory:Participate/Script aid"

From Free Software Directory
Jump to: navigation, search
(Added print-ERE option, allowing per-file match view using any text editor.)
(Use spaces instead of tabs, rename print-ERE to print-ere.)
Line 38: Line 38:
 
Now leave the script to do its work and wait for the sound clue to continue working on the evaluation.
 
Now leave the script to do its work and wait for the sound clue to continue working on the evaluation.
  
After this, you might be wondering: how to go exactly to the points where the script found something, on each file? For this purpose there is the <code>--print-ERE</code> (<code>-p</code>) option. It prints an extended regular expression (ERE) that can be reused by you in any text editor to look for the same things, per file.
+
After this, you might be wondering: how to go exactly to the points where the script found something, on each file? For this purpose there is the <code>--print-ere</code> (<code>-p</code>) option. It prints an extended regular expression (ERE) that can be reused by you in any text editor to look for the same things, per file.
  
 
For example, you can do this:
 
For example, you can do this:
Line 52: Line 52:
 
#!/bin/bash
 
#!/bin/bash
  
# FSD Script Aid.sh: Helps user evaluate entries for the Free Software Directory.
+
#   FSD Script Aid: Helps evaluate entries for Free Software Directory.
# Copyright (C) 2016, 2018, 2020, 2021  Adonay "adfeno" Felipe Nogueira <https://libreplanet.org/wiki/User:Adfeno> <adfeno@hyperbola.info>
+
#   Copyright (C) 2016, 2018, 2020, 2021  Adonay "adfeno" Felipe Nogueira
 
+
#                              <https://libreplanet.org/wiki/User:Adfeno>
# This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+
#                                                  <adfeno@hyperbola.info>
 
+
#
# This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU Affero General Public License for more details.
+
#   This program is free software: you can redistribute it and/or modify
 
+
#    it under the terms of the GNU Affero General Public License as published by
# You should have received a copy of the GNU Affero General Public License along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
#    the Free Software Foundation, either version 3 of the License, or
 
+
#    (at your option) any later version.
 
+
#
# For more information on the purpose and caveats of this script, see <https://directory.fsf.org/wiki/Free_Software_Directory:Participate/Script_aid>.
+
#   This program is distributed in the hope that it will be useful,
 +
#    but WITHOUT ANY WARRANTY; without even the implied warranty of
 +
#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 +
#    GNU Affero General Public License for more details.
 +
#
 +
#   You should have received a copy of the GNU Affero General Public License
 +
#    along with this program.  If not, see <https://www.gnu.org/licenses/>.
  
  
 
# # Dependencies
 
# # Dependencies
 
+
#
 
+
#
 
# * GNU bash;
 
# * GNU bash;
 
# * any implementation of the following POSIX utilities:
 
# * any implementation of the following POSIX utilities:
# * file, whose -i option prints MIME type and character set;
+
#   * file, whose -i option prints MIME type and character set;
# * find;
+
#   * find;
# * grep;
+
#   * grep;
# * printf;
+
#   * printf;
# * sed;
+
#   * sed;
# * tr.
+
#   * tr.
 
+
#
 
+
#
 
# # Usage
 
# # Usage
 
+
#
 
+
#
 
# It's recommended to grant executable permisisons for your user, like so:
 
# It's recommended to grant executable permisisons for your user, like so:
 
+
#
 
# chmod u+x [path to FSD Script Aid.sh]
 
# chmod u+x [path to FSD Script Aid.sh]
 
+
#
 
# Then, change to a directory containing the source files.
 
# Then, change to a directory containing the source files.
 
+
#
 
# Now do:
 
# Now do:
 
+
#
 
# [path to FSD Script Aid.sh] [options] > [path to the output .CSV file]
 
# [path to FSD Script Aid.sh] [options] > [path to the output .CSV file]
 
+
#
 
# Some options prevent normal output, so you could just do:
 
# Some options prevent normal output, so you could just do:
 
+
#
 
# [path to FSD Script Aid.sh] [options]
 
# [path to FSD Script Aid.sh] [options]
 
+
#
 
+
#
 
# # Options
 
# # Options
 
+
#
 
+
#
# ## '--print-ERE' ('-p')
+
# ## '--print-ere' ('-p')
 
+
#
# Prevent normal output, and print an extended regular expression (ERE) that can be reused for checks against any file.
+
# Prevent normal output, and print an extended regular expression (ERE) that
 
+
# can be reused for checks against any file.
# This can be reused against each file with a text editor's regular expression search.
+
#
 
+
# This can be reused against each file with a text editor's regular
 +
# expression search.
 +
#
 
# For example, with GNU less, one can do:
 
# For example, with GNU less, one can do:
 
+
#
 
# less -p "$([path to FSD Script Aid.sh] -p)" "file1" "file2" "fileN"
 
# less -p "$([path to FSD Script Aid.sh] -p)" "file1" "file2" "fileN"
 
+
#
# … and then press N to go to each next match in the first file. Shift + N searches backwards in the same file.
+
# … and then press N to go to each next match in the first file. Shift + N
 +
# searches backwards in the same file.
  
  
Line 116: Line 125:
  
 
# Copyright symbol
 
# Copyright symbol
FSD_SCRIPT_AID_LICENSING_ERE="©"
+
licensing_ere="©"
  
 
# Use GNU bash's += assignment to append to an existing variable.
 
# Use GNU bash's += assignment to append to an existing variable.
 +
#
 
# Agreement
 
# Agreement
FSD_SCRIPT_AID_LICENSING_ERE+="|agreement|a(co|cue)rdo"
+
licensing_ere+="|agreement|a(co|cue)rdo"
 
# Allowed
 
# Allowed
FSD_SCRIPT_AID_LICENSING_ERE+="|allowed|permitid[ao]"
+
licensing_ere+="|allowed|permitid[ao]"
 
# As is
 
# As is
FSD_SCRIPT_AID_LICENSING_ERE+="|as[[:space:]-]*is[^[:alnum:]]"
+
licensing_ere+="|as[[:space:]-]*is[^[:alnum:]]"
 
# Condition
 
# Condition
FSD_SCRIPT_AID_LICENSING_ERE+="|condi(tions?|ci(ón|ones)|ç(ão|ões))"
+
licensing_ere+="|condi(tions?|ci(ón|ones)|ç(ão|ões))"
 
# Copyright, copyleft, copyfarleft, copyfair, copymi
 
# Copyright, copyleft, copyfarleft, copyfair, copymi
FSD_SCRIPT_AID_LICENSING_ERE+="|copy(right|(far)?left|fai?r|m[ei])"
+
licensing_ere+="|copy(right|(far)?left|fai?r|m[ei])"
 
# EULA, exclusive
 
# EULA, exclusive
FSD_SCRIPT_AID_LICENSING_ERE+="|eula|exclusiv[aeo]"
+
licensing_ere+="|eula|exclusiv[aeo]"
 
# Forbid
 
# Forbid
FSD_SCRIPT_AID_LICENSING_ERE+="|forbid(s|den)?|pro(hibited|h?ibid[ao])"
+
licensing_ere+="|forbid(s|den)?|pro(hibited|h?ibid[ao])"
 
# License abbreviations
 
# License abbreviations
FSD_SCRIPT_AID_LICENSING_ERE+="|[al]?gpl|fdl"
+
licensing_ere+="|[al]?gpl|fdl"
 
# Law
 
# Law
FSD_SCRIPT_AID_LICENSING_ERE+="|l(aw|e[iy])"
+
licensing_ere+="|l(aw|e[iy])"
 
# Liable
 
# Liable
FSD_SCRIPT_AID_LICENSING_ERE+="|liab(le|ilit(y|ies))"
+
licensing_ere+="|liab(le|ilit(y|ies))"
 
# Responsible
 
# Responsible
FSD_SCRIPT_AID_LICENSING_ERE+="|respons(ib(le|ilit(y|ies))|ab(le|ilidad(es?)?)|áve(l|is))"
+
licensing_ere+="|respons(ib(le|ilit(y|ies))|ab(le|ilidad(es?)?)|áve(l|is))"
 
# License
 
# License
FSD_SCRIPT_AID_LICENSING_ERE+="|licen([cs]e|(ç|ci)a)"
+
licensing_ere+="|licen([cs]e|(ç|ci)a)"
 
# Notice
 
# Notice
FSD_SCRIPT_AID_LICENSING_ERE+="|not(ice|[ií]cia|ifica(tion|ção|ción))"
+
licensing_ere+="|not(ice|[ií]cia|ifica(tion|ção|ción))"
 
# Patent, right
 
# Patent, right
FSD_SCRIPT_AID_LICENSING_ERE+="|patente?|right|d(erech|ireit)o|droit"
+
licensing_ere+="|patente?|right|d(erech|ireit)o|droit"
 
# Terms
 
# Terms
FSD_SCRIPT_AID_LICENSING_ERE+="|t(erms?|érminos?|ermos?)"
+
licensing_ere+="|t(erms?|érminos?|ermos?)"
 
# Trade
 
# Trade
FSD_SCRIPT_AID_LICENSING_ERE+="|trade[[:space:]]+(mark|secret)"
+
licensing_ere+="|trade[[:space:]]+(mark|secret)"
 
# Transfer, warrant
 
# Transfer, warrant
FSD_SCRIPT_AID_LICENSING_ERE+="|transfer|gu?arant|warrant"
+
licensing_ere+="|transfer|gu?arant|warrant"
  
  
Line 159: Line 169:
  
 
# Start of boundary
 
# Start of boundary
FSD_SCRIPT_AID_JAVASCRIPT_ERE="(^|[^[:alnum:]]+)"
+
javascript_ere="(^|[^[:alnum:]]+)"
FSD_SCRIPT_AID_JAVASCRIPT_ERE+="("
+
javascript_ere+="("
  
 
# Script tag or addEventListener() function
 
# Script tag or addEventListener() function
FSD_SCRIPT_AID_JAVASCRIPT_ERE+="script|addeventlistener"
+
javascript_ere+="script|addeventlistener"
  
 
# Start of after, before, on variations
 
# Start of after, before, on variations
FSD_SCRIPT_AID_JAVASCRIPT_ERE+="|(after|before|on)"
+
javascript_ere+="|(after|before|on)"
FSD_SCRIPT_AID_JAVASCRIPT_ERE+="("
+
javascript_ere+="("
  
 
# Abort, autocomplete, blur, cancel, canplay
 
# Abort, autocomplete, blur, cancel, canplay
FSD_SCRIPT_AID_JAVASCRIPT_ERE+="abort|autocomplete(error)?|blur|cancel|canplay(through)?"
+
javascript_ere+="abort|autocomplete(error)?|blur|cancel|canplay(through)?"
 
# ( Cue, duration, hash, language, rate, readystate, volume ) … change
 
# ( Cue, duration, hash, language, rate, readystate, volume ) … change
FSD_SCRIPT_AID_JAVASCRIPT_ERE+="|(cue|duration|hash|language|rate|readystate|volume)?change"
+
javascript_ere+="|(cue|duration|hash|language|rate|readystate|volume)?change"
 
# Click, close, contextmenu
 
# Click, close, contextmenu
FSD_SCRIPT_AID_JAVASCRIPT_ERE+="|(db)?click|close|contextmenu"
+
javascript_ere+="|(db)?click|close|contextmenu"
 
# Drag and drop
 
# Drag and drop
FSD_SCRIPT_AID_JAVASCRIPT_ERE+="|drag(end|enter|exit|leave|over|start)|drop"
+
javascript_ere+="|drag(end|enter|exit|leave|over|start)|drop"
 
# Emptied, ended, error, focus, input, invalid
 
# Emptied, ended, error, focus, input, invalid
FSD_SCRIPT_AID_JAVASCRIPT_ERE+="|emptied|ended|error|focus|input|invalid"
+
javascript_ere+="|emptied|ended|error|focus|input|invalid"
 
# Key presses
 
# Key presses
FSD_SCRIPT_AID_JAVASCRIPT_ERE+="|key(down|press|up)"
+
javascript_ere+="|key(down|press|up)"
 
# Load
 
# Load
FSD_SCRIPT_AID_JAVASCRIPT_ERE+="|(un)?load(ed(meta)?data)?"
+
javascript_ere+="|(un)?load(ed(meta)?data)?"
 
# Start, message
 
# Start, message
FSD_SCRIPT_AID_JAVASCRIPT_ERE+="|start|message"
+
javascript_ere+="|start|message"
 
# Mouse
 
# Mouse
FSD_SCRIPT_AID_JAVASCRIPT_ERE+="|mouse(down|enter|leave|move|out|over|up|wheel)"
+
javascript_ere+="|mouse(down|enter|leave|move|out|over|up|wheel)"
 
# Connectivity and page
 
# Connectivity and page
FSD_SCRIPT_AID_JAVASCRIPT_ERE+="|(off|on)line|page(hide|show)"
+
javascript_ere+="|(off|on)line|page(hide|show)"
 
# Play, popstate, print, progress
 
# Play, popstate, print, progress
FSD_SCRIPT_AID_JAVASCRIPT_ERE+="|pause|play(ing)?|popstate|print|progress"
+
javascript_ere+="|pause|play(ing)?|popstate|print|progress"
 
# Reset, resize, scroll, seek
 
# Reset, resize, scroll, seek
FSD_SCRIPT_AID_JAVASCRIPT_ERE+="|reset|resize|scroll|seek(ed|ing)"
+
javascript_ere+="|reset|resize|scroll|seek(ed|ing)"
 
# Select, show, sort, stalled, storage, submit
 
# Select, show, sort, stalled, storage, submit
FSD_SCRIPT_AID_JAVASCRIPT_ERE+="|select|show|sort|stalled|storage|submit"
+
javascript_ere+="|select|show|sort|stalled|storage|submit"
 
# Suspend, timeupdate, toggle, waiting
 
# Suspend, timeupdate, toggle, waiting
FSD_SCRIPT_AID_JAVASCRIPT_ERE+="|suspend|timeupdate|toggle|waiting"
+
javascript_ere+="|suspend|timeupdate|toggle|waiting"
  
 
# End of after, before, on variations
 
# End of after, before, on variations
FSD_SCRIPT_AID_JAVASCRIPT_ERE+=")"
+
javascript_ere+=")"
  
 
# End of boundary
 
# End of boundary
FSD_SCRIPT_AID_JAVASCRIPT_ERE+=")"
+
javascript_ere+=")"
FSD_SCRIPT_AID_JAVASCRIPT_ERE+="([^[:alnum:]]+|$)"
+
javascript_ere+="([^[:alnum:]]+|$)"
  
  
Line 210: Line 220:
  
 
# Google
 
# Google
FSD_SCRIPT_AID_OTHER_ERE+="gcm|google([^[:alnum:]]*cloud[^[:alnum:]]*messaging)?|youtube|yt"
+
other_ere+="gcm|google([^[:alnum:]]*cloud[^[:alnum:]]*messaging)?|youtube|yt"
 
# Microsoft, Facebook, WhatsApp, Telegram
 
# Microsoft, Facebook, WhatsApp, Telegram
FSD_SCRIPT_AID_OTHER_ERE+="|microsoft|facebook|whats[^[:alnum:]]*app|telegram"
+
other_ere+="|microsoft|facebook|whats[^[:alnum:]]*app|telegram"
 
# CDNs, Amazon, CloudFlare
 
# CDNs, Amazon, CloudFlare
FSD_SCRIPT_AID_OTHER_ERE+="|amazon|aws|cloud[^[:alnum:]]*flare|cdn"
+
other_ere+="|amazon|aws|cloud[^[:alnum:]]*flare|cdn"
 
# Apple, Uber, AirBnB
 
# Apple, Uber, AirBnB
FSD_SCRIPT_AID_OTHER_ERE+="|apple|uber|air[^[:alnum:]]*bnb"
+
other_ere+="|apple|uber|air[^[:alnum:]]*bnb"
 
# System distributions
 
# System distributions
FSD_SCRIPT_AID_OTHER_ERE+="|android|lineage|cyanogen"
+
other_ere+="|android|lineage|cyanogen"
 
# DRM, Chrome and derivated
 
# DRM, Chrome and derivated
FSD_SCRIPT_AID_OTHER_ERE+="|drm|chrom(e|ium)|electron"
+
other_ere+="|drm|chrom(e|ium)|electron"
  
if [[ $1 == "--print-ERE" ||
+
if [[ $1 == "--print-ere" ||
$1 == "-p" ]]; then
+
  $1 == "-p" ]]; then
  
# Print the extended regular expressions so that you can use then later.
+
  # Print the extended regular expressions so that you can use then later.
echo "$FSD_SCRIPT_AID_LICENSING_ERE|$FSD_SCRIPT_AID_JAVASCRIPT_ERE|$FSD_SCRIPT_AID_OTHER_ERE"
+
  echo "$licensing_ere|$javascript_ere|$other_ere"
exit
+
  exit
 
fi
 
fi
  
 +
# Separates MIME type from the character set.
 +
#
 +
# * Arguments
 +
#  * $1: file path to inspect MIME type and character set.
 +
# * Standard output
 +
#  * Comma-separated pair consisting of MIME type and character set.
 
separate_mime_from_charset() {
 
separate_mime_from_charset() {
# * Arguments
+
# Make use of -i to get MIME type and character set.
# * $1: file path to inspect MIME type and character set.
+
# sed does the split and tr deletes line feeds/new lines.
# * Standard output
+
  file -hi "$1" \
# * Comma-separated pair consisting of MIME type and character set.
+
    | sed \
 
+
      '{
# Make use of -i to get MIME type and character set.
+
        s/^\.\//"/g
# sed does the split and tr deletes line feeds/new lines.
+
        s/: /",/g
file -hi "$1" \
+
        s/; charset=/,/g
| sed \
+
      }' \
'{
+
      | tr -d "\n"
s/^\.\//"/g
 
s/: /",/g
 
s/; charset=/,/g
 
}' \
 
| tr -d "\n"
 
 
}
 
}
  
 +
# Tells if ERE was found in file.
 +
#
 +
# * Arguments
 +
#  * $1: extended regular expression to be searched;
 +
#  * $2: file path to look for given extended regular expression.
 +
# * Standard output
 +
#  * If a matche was found
 +
#    * String "TRUE"
 +
#  * Else
 +
#    * String "FALSE"
 
was_ere_found() {
 
was_ere_found() {
# * Arguments
 
# * $1: extended regular expression to be searched;
 
# * $2: file path to look for given extended regular expression.
 
# * Standard output
 
# * If a matche was found
 
# * String "TRUE"
 
# * Else
 
# * String "FALSE"
 
  
 
grep -Eiq "$1" "$2"
 
grep -Eiq "$1" "$2"
Line 277: Line 290:
 
-exec printf ',' \; \
 
-exec printf ',' \; \
 
-exec bash -c \
 
-exec bash -c \
'was_ere_found "'"$FSD_SCRIPT_AID_LICENSING_ERE"'" "{}"' \; \
+
'was_ere_found "'"$licensing_ere"'" "{}"' \; \
 
-exec printf ',' \; \
 
-exec printf ',' \; \
 
-exec bash -c \
 
-exec bash -c \
'was_ere_found "'"$FSD_SCRIPT_AID_JAVASCRIPT_ERE"'" "{}"' \; \
+
'was_ere_found "'"$javascript_ere"'" "{}"' \; \
 
-exec printf ',' \; \
 
-exec printf ',' \; \
 
-exec bash -c \
 
-exec bash -c \
'was_ere_found "'"$FSD_SCRIPT_AID_OTHER_ERE"'" "{}"' \; \
+
'was_ere_found "'"$other_ere"'" "{}"' \; \
 
-exec printf ',' \; \
 
-exec printf ',' \; \
 
-exec printf '\r\n' \;
 
-exec printf '\r\n' \;
 
</pre>
 
</pre>

Revision as of 12:09, 26 February 2021

Purpose

This script aims to help contributors on finding things that might be needed to consider during evaluation of the project/software.

Besides searching for common licensing terms, it lists all the MIME/media types of files in the current directory, so that one can find cases of files without complete and corresponding source and also find JavaScript files to look or insert GNU LibreJS syntax.

To tackle cases of files which are partly JavaScript, it also scans for words related to JS and event handlers.

As an extra precaution, an extra pass for other possible problematic words is made.

There is also a field for notes which is always left empty for the evaluator to do the appropriate observations or even insert marks of continuation for future resume of the review.

The output is in CSV format, making it suitable for parsing by other software, such as LibreOffice Calc, GNU R and even GNU Awk. For the last case, before doing so, set FPAT to ([^,]*)|(\"[^\"]+\") and RS to \r\n, This variable setting was based on the related section in the GNU Awk User's Guide and on Awk's Texinfo/Info page.

In all cases, when importing, make sure that the selected language is English, so that "TRUE" and "FALSE" can be translated as correct boolean representation in your language of choice. Also, be aware of false-positives.

You're welcome to contribute to this script and add your name and contact information to the copyright notice of the script.

Usage

First, it's recommended to grant executable permissisns for your user, like so:

chmod u+x [path to FSD Script Aid.sh]

Then, take the complete corresponding source of the project being evaluated (e.g.: when using git clone, you can accomplish this using the --recursive option.

git clone --recursive [Some git repository.]
cd [Directory created by git]
[Script aid.] > [Desired text file to store output.]; printf '\a'

printf '\a' can be replaced by a command to play an audio file of your choice.

Now leave the script to do its work and wait for the sound clue to continue working on the evaluation.

After this, you might be wondering: how to go exactly to the points where the script found something, on each file? For this purpose there is the --print-ere (-p) option. It prints an extended regular expression (ERE) that can be reused by you in any text editor to look for the same things, per file.

For example, you can do this:

less -p "$([Script aid.] -p)" "file1" "file2" "fileN"


Script

#!/bin/bash

#    FSD Script Aid: Helps evaluate entries for Free Software Directory.
#    Copyright (C) 2016, 2018, 2020, 2021  Adonay "adfeno" Felipe Nogueira
#                               <https://libreplanet.org/wiki/User:Adfeno>
#                                                  <adfeno@hyperbola.info>
#
#    This program is free software: you can redistribute it and/or modify
#    it under the terms of the GNU Affero General Public License as published by
#    the Free Software Foundation, either version 3 of the License, or
#    (at your option) any later version.
#
#    This program is distributed in the hope that it will be useful,
#    but WITHOUT ANY WARRANTY; without even the implied warranty of
#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#    GNU Affero General Public License for more details.
#
#    You should have received a copy of the GNU Affero General Public License
#    along with this program.  If not, see <https://www.gnu.org/licenses/>.


# # Dependencies
#
#
# * GNU bash;
# * any implementation of the following POSIX utilities:
#   * file, whose -i option prints MIME type and character set;
#   * find;
#   * grep;
#   * printf;
#   * sed;
#   * tr.
#
#
# # Usage
#
#
# It's recommended to grant executable permisisons for your user, like so:
#
# chmod u+x [path to FSD Script Aid.sh]
#
# Then, change to a directory containing the source files.
#
# Now do:
#
# [path to FSD Script Aid.sh] [options] > [path to the output .CSV file]
#
# Some options prevent normal output, so you could just do:
#
# [path to FSD Script Aid.sh] [options]
#
#
# # Options
#
#
# ## '--print-ere' ('-p')
#
# Prevent normal output, and print an extended regular expression (ERE) that
# can be reused for checks against any file.
#
# This can be reused against each file with a text editor's regular
# expression search.
#
# For example, with GNU less, one can do:
#
# less -p "$([path to FSD Script Aid.sh] -p)" "file1" "file2" "fileN"
#
# … and then press N to go to each next match in the first file. Shift + N
# searches backwards in the same file.


# # Extended regular expressions (ERE) to match possible licensing issues


# Copyright symbol
licensing_ere="©"

# Use GNU bash's += assignment to append to an existing variable.
#
# Agreement
licensing_ere+="|agreement|a(co|cue)rdo"
# Allowed
licensing_ere+="|allowed|permitid[ao]"
# As is
licensing_ere+="|as[[:space:]-]*is[^[:alnum:]]"
# Condition
licensing_ere+="|condi(tions?|ci(ón|ones)|ç(ão|ões))"
# Copyright, copyleft, copyfarleft, copyfair, copymi
licensing_ere+="|copy(right|(far)?left|fai?r|m[ei])"
# EULA, exclusive
licensing_ere+="|eula|exclusiv[aeo]"
# Forbid
licensing_ere+="|forbid(s|den)?|pro(hibited|h?ibid[ao])"
# License abbreviations
licensing_ere+="|[al]?gpl|fdl"
# Law
licensing_ere+="|l(aw|e[iy])"
# Liable
licensing_ere+="|liab(le|ilit(y|ies))"
# Responsible
licensing_ere+="|respons(ib(le|ilit(y|ies))|ab(le|ilidad(es?)?)|áve(l|is))"
# License
licensing_ere+="|licen([cs]e|(ç|ci)a)"
# Notice
licensing_ere+="|not(ice|[ií]cia|ifica(tion|ção|ción))"
# Patent, right
licensing_ere+="|patente?|right|d(erech|ireit)o|droit"
# Terms
licensing_ere+="|t(erms?|érminos?|ermos?)"
# Trade
licensing_ere+="|trade[[:space:]]+(mark|secret)"
# Transfer, warrant
licensing_ere+="|transfer|gu?arant|warrant"


# # Extended regular expression (ERE) to match JavaScript issues


# Start of boundary
javascript_ere="(^|[^[:alnum:]]+)"
javascript_ere+="("

# Script tag or addEventListener() function
javascript_ere+="script|addeventlistener"

# Start of after, before, on variations
javascript_ere+="|(after|before|on)"
javascript_ere+="("

# Abort, autocomplete, blur, cancel, canplay
javascript_ere+="abort|autocomplete(error)?|blur|cancel|canplay(through)?"
# ( Cue, duration, hash, language, rate, readystate, volume ) … change
javascript_ere+="|(cue|duration|hash|language|rate|readystate|volume)?change"
# Click, close, contextmenu
javascript_ere+="|(db)?click|close|contextmenu"
# Drag and drop
javascript_ere+="|drag(end|enter|exit|leave|over|start)|drop"
# Emptied, ended, error, focus, input, invalid
javascript_ere+="|emptied|ended|error|focus|input|invalid"
# Key presses
javascript_ere+="|key(down|press|up)"
# Load
javascript_ere+="|(un)?load(ed(meta)?data)?"
# Start, message
javascript_ere+="|start|message"
# Mouse
javascript_ere+="|mouse(down|enter|leave|move|out|over|up|wheel)"
# Connectivity and page
javascript_ere+="|(off|on)line|page(hide|show)"
# Play, popstate, print, progress
javascript_ere+="|pause|play(ing)?|popstate|print|progress"
# Reset, resize, scroll, seek
javascript_ere+="|reset|resize|scroll|seek(ed|ing)"
# Select, show, sort, stalled, storage, submit
javascript_ere+="|select|show|sort|stalled|storage|submit"
# Suspend, timeupdate, toggle, waiting
javascript_ere+="|suspend|timeupdate|toggle|waiting"

# End of after, before, on variations
javascript_ere+=")"

# End of boundary
javascript_ere+=")"
javascript_ere+="([^[:alnum:]]+|$)"


# # Extended regular expression (ERE) to match possible other issues


# Google
other_ere+="gcm|google([^[:alnum:]]*cloud[^[:alnum:]]*messaging)?|youtube|yt"
# Microsoft, Facebook, WhatsApp, Telegram
other_ere+="|microsoft|facebook|whats[^[:alnum:]]*app|telegram"
# CDNs, Amazon, CloudFlare
other_ere+="|amazon|aws|cloud[^[:alnum:]]*flare|cdn"
# Apple, Uber, AirBnB
other_ere+="|apple|uber|air[^[:alnum:]]*bnb"
# System distributions
other_ere+="|android|lineage|cyanogen"
# DRM, Chrome and derivated
other_ere+="|drm|chrom(e|ium)|electron"

if [[ $1 == "--print-ere" ||
  $1 == "-p" ]]; then

  # Print the extended regular expressions so that you can use then later.
  echo "$licensing_ere|$javascript_ere|$other_ere"
  exit
fi

# Separates MIME type from the character set.
#
# * Arguments
#   * $1: file path to inspect MIME type and character set.
# * Standard output
#   * Comma-separated pair consisting of MIME type and character set.
separate_mime_from_charset() {
# Make use of -i to get MIME type and character set.
# sed does the split and tr deletes line feeds/new lines.
  file -hi "$1" \
    | sed \
      '{
         s/^\.\//"/g
         s/: /",/g
         s/; charset=/,/g
       }' \
      | tr -d "\n"
}

# Tells if ERE was found in file.
#
# * Arguments
#   * $1: extended regular expression to be searched;
#   * $2: file path to look for given extended regular expression.
# * Standard output
#   * If a matche was found
#     * String "TRUE"
#   * Else
#     * String "FALSE"
was_ere_found() {

	grep -Eiq "$1" "$2"
	[ $? -eq 0 ] && printf "TRUE" || printf "FALSE"
}

# Use GNU bash's -f option to export functions for use with find.
export -f separate_mime_from_charset
export -f was_ere_found

printf '"Path","MIME","Charset","Licensing","JavaScript","Other","Notes"\r\n'
find "." \
	\( \
		-type d \
		\( -name '.cvs' -o -name '.hg' -o -name '.git' -o -name '.svn' \) \
	\) -prune -o \
	! -type d \
	-exec bash -c \
		'separate_mime_from_charset "{}"' \; \
	-exec printf ',' \; \
	-exec bash -c \
		'was_ere_found "'"$licensing_ere"'" "{}"' \; \
	-exec printf ',' \; \
	-exec bash -c \
		'was_ere_found "'"$javascript_ere"'" "{}"' \; \
	-exec printf ',' \; \
	-exec bash -c \
		'was_ere_found "'"$other_ere"'" "{}"' \; \
	-exec printf ',' \; \
	-exec printf '\r\n' \;


Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the page “GNU Free Documentation License”.

The copyright and license notices on this page only apply to the text on this page. Any software or copyright-licenses or other similar notices described in this text has its own copyright notice and license, which can usually be found in the distribution or license text itself.