Difference between revisions of "Free Software Directory:Pywikibot"
(→pywikibot) |
|||
| (63 intermediate revisions by the same user not shown) | |||
| Line 6: | Line 6: | ||
<pre> | <pre> | ||
# Allow batch uploading for pywikibot | # Allow batch uploading for pywikibot | ||
| + | $wgGrantPermissions['editpage']['approverevisions'] = true; # Required by the MediaWiki Extension "Approved Revs" to create new pages | ||
$wgGrantPermissions['editprotected']['editarticles'] = true; | $wgGrantPermissions['editprotected']['editarticles'] = true; | ||
| + | $wgFileExtensions[] = 'wiki'; | ||
</pre> | </pre> | ||
| Line 18: | Line 20: | ||
====Background to using the bot==== | ====Background to using the bot==== | ||
| + | The method of using BotPasswords improves the security overall. However bot's are required due to a CASAuth issue: | ||
| + | |||
| + | See also the '''CASAuth''' issue on this page. | ||
| + | |||
| + | ==pywikibot== | ||
| + | <pre> | ||
| + | pywikibot_dir="pywikibot-core"; | ||
| + | site="http://directory.fsf.org/wiki/Main_Page"; | ||
| + | family="freesoftwaredirectory"; | ||
| + | |||
| + | # The scripts/pagefromfile.py requires scripts/i18n/ (eg scripts/i18n/pagefromfile/) that is pulled with the --recursive option (or "cd pywikibot-core && git submodule update --init" after git clone) | ||
| + | git clone --recursive https://gerrit.wikimedia.org/r/pywikibot/core "$pywikibot_dir" | ||
| + | cd "$pywikibot_dir" | ||
| + | python generate_family_file.py "$site" "$family" | ||
| + | python pwb.py generate_user_files | ||
| + | # Select family of sites we are working on, just enter the number or name (default: wikipedia): freesoftwaredirectory | ||
| + | # The only known language: en | ||
| + | # The language code of the site we're working on (default: en): | ||
| + | # Username on en:freesoftwaredirectory: WikiSysop | ||
| + | # Do you want to add any other projects? ([y]es, [N]o): n | ||
| + | # Do you want to add a BotPassword for WikiSysop? ([y]es, [N]o, [q]uit): y | ||
| + | # See https://www.mediawiki.org/wiki/Manual:Pywikibot/BotPasswords to | ||
| + | # know how to get codes.Please note that plain text in /home/root | ||
| + | # /pywikibot-core/user-password.py and anyone with read access to that | ||
| + | # directory will be able read the file. | ||
| + | # BotPassword's "bot name" for WikiSysop: wikibot | ||
| + | # BotPassword's "password" for "WikiSysop" (no characters will be shown): | ||
| + | # '/home/root/pywikibot-core/user-config.py' written. | ||
| + | </pre> | ||
| + | |||
| + | ==Test pywikibot on MediaWiki== | ||
| + | python pwb.py upload -always -ignorewarn ~/heckert_gnu.transp.small.png "Some description" | ||
| + | |||
| + | ===Estimate time for batch uploading=== | ||
| + | <pre> | ||
| + | script_generated=$(date +"%Y-%m-%d %H:%M:%S.%N"); # .wiki files with identical content since the last revision will be ignored ("No changes were needed on [[YOURTITLE]]") | ||
| + | SECONDS=0 | ||
| + | |||
| + | for (( i=0; i<10; i++ )); do | ||
| + | echo "$script_generated" > Test.wiki; | ||
| + | python pwb.py pagefromfile -summary:"The edit summary" -title:"Test$i" -file:"Test.wiki" -textonly -force -pt:0 | ||
| + | done | ||
| + | duration=$SECONDS | ||
| + | echo "$(($duration / 10)) seconds per entry in average." | ||
| + | rm -f Test.wiki | ||
| + | </pre> | ||
| + | |||
| + | ==Issues== | ||
| + | |||
| + | ===CASAuth=== | ||
| + | This issue doesn't have to be fixed since there's a work around, but it would be great if it were fixed to give us more options. | ||
| + | |||
zhuyifei1999: | zhuyifei1999: | ||
| Line 39: | Line 93: | ||
irc.freenode.net, http://commons.wikimedia.org/wiki/User:Zhuyifei1999 | irc.freenode.net, http://commons.wikimedia.org/wiki/User:Zhuyifei1999 | ||
| + | ==Suggestions== | ||
| − | == | + | ===OAuth=== |
| − | + | pywikibot can be used with OAuth - https://www.mediawiki.org/wiki/Manual:Pywikibot/OAuth. However, the OAuth extension is currently not installed - https://directory.fsf.org/wiki/Special:Version | |
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
==Further reading== | ==Further reading== | ||
Read https://www.mediawiki.org/wiki/Manual:Importing_XML_dumps#Using_pywikibot,_pagefromfile.py_and_Nokogiri for more information | Read https://www.mediawiki.org/wiki/Manual:Importing_XML_dumps#Using_pywikibot,_pagefromfile.py_and_Nokogiri for more information | ||
Latest revision as of 06:27, 17 August 2018
Free Software Directory:Participate/Using pywikibot should be merged to this page.
Contents
MediaWiki
- Add this to LocalSettings.php
# Allow batch uploading for pywikibot $wgGrantPermissions['editpage']['approverevisions'] = true; # Required by the MediaWiki Extension "Approved Revs" to create new pages $wgGrantPermissions['editprotected']['editarticles'] = true; $wgFileExtensions[] = 'wiki';
Generate a bot account for your user
The bot will be associated with the user name that are logged in.
- Login as WikiSysop in http://directory.fsf.org/
- Visit http://directory.fsf.org/wiki/Special:BotPasswords and generate a new bot.
- Bot name: wikibot
- Store the generated password somewhere
Background to using the bot
The method of using BotPasswords improves the security overall. However bot's are required due to a CASAuth issue:
See also the CASAuth issue on this page.
pywikibot
pywikibot_dir="pywikibot-core"; site="http://directory.fsf.org/wiki/Main_Page"; family="freesoftwaredirectory"; # The scripts/pagefromfile.py requires scripts/i18n/ (eg scripts/i18n/pagefromfile/) that is pulled with the --recursive option (or "cd pywikibot-core && git submodule update --init" after git clone) git clone --recursive https://gerrit.wikimedia.org/r/pywikibot/core "$pywikibot_dir" cd "$pywikibot_dir" python generate_family_file.py "$site" "$family" python pwb.py generate_user_files # Select family of sites we are working on, just enter the number or name (default: wikipedia): freesoftwaredirectory # The only known language: en # The language code of the site we're working on (default: en): # Username on en:freesoftwaredirectory: WikiSysop # Do you want to add any other projects? ([y]es, [N]o): n # Do you want to add a BotPassword for WikiSysop? ([y]es, [N]o, [q]uit): y # See https://www.mediawiki.org/wiki/Manual:Pywikibot/BotPasswords to # know how to get codes.Please note that plain text in /home/root # /pywikibot-core/user-password.py and anyone with read access to that # directory will be able read the file. # BotPassword's "bot name" for WikiSysop: wikibot # BotPassword's "password" for "WikiSysop" (no characters will be shown): # '/home/root/pywikibot-core/user-config.py' written.
Test pywikibot on MediaWiki
python pwb.py upload -always -ignorewarn ~/heckert_gnu.transp.small.png "Some description"
Estimate time for batch uploading
script_generated=$(date +"%Y-%m-%d %H:%M:%S.%N"); # .wiki files with identical content since the last revision will be ignored ("No changes were needed on [[YOURTITLE]]")
SECONDS=0
for (( i=0; i<10; i++ )); do
echo "$script_generated" > Test.wiki;
python pwb.py pagefromfile -summary:"The edit summary" -title:"Test$i" -file:"Test.wiki" -textonly -force -pt:0
done
duration=$SECONDS
echo "$(($duration / 10)) seconds per entry in average."
rm -f Test.wiki
Issues
CASAuth
This issue doesn't have to be fixed since there's a work around, but it would be great if it were fixed to give us more options.
zhuyifei1999:
Pywikibot [1] is a MediaWiki bot library which we are attemting to utilize to perform batch uploads. However, because of CASAuth, we are unable [2] to perform main account CAS-based login, and have to resort to using BotPasswords. However, to upload, permissions to alter "File" namespace is needed, but it is restricted to the 'editarticles' permission [4], and there is no grant [5] that gives the permission to any BotPasswords-based login. Please add 'editarticles' to the 'Edit protected pages' set [5], or any other set if you deem appropriate, to allow batch uploading.
- https://www.mediawiki.org/wiki/Manual:Pywikibot
- https://github.com/CWRUChielLab/CASAuth/issues/7
- https://www.mediawiki.org/wiki/Manual:Pywikibot/BotPasswords
- https://directory.fsf.org/wiki/Special:ListGroupRights
- https://directory.fsf.org/wiki/Special:ListGrants
// zhuyifei1999@gmail.com, zhuyifei1999_ in #pywikibot at irc.freenode.net, http://commons.wikimedia.org/wiki/User:Zhuyifei1999
Suggestions
OAuth
pywikibot can be used with OAuth - https://www.mediawiki.org/wiki/Manual:Pywikibot/OAuth. However, the OAuth extension is currently not installed - https://directory.fsf.org/wiki/Special:Version
Further reading
Read https://www.mediawiki.org/wiki/Manual:Importing_XML_dumps#Using_pywikibot,_pagefromfile.py_and_Nokogiri for more information
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the page “GNU Free Documentation License”.
The copyright and license notices on this page only apply to the text on this page. Any software or copyright-licenses or other similar notices described in this text has its own copyright notice and license, which can usually be found in the distribution or license text itself.