⚓ T308137 Deploy "add a link" to 12th round of wikis
Article Images
Deploy "add a link" to 12th round of wikis
Deploy "add a link" to 12th round of wikis
- Training models
- Eastern Mari Wikipedia mhr
- Maori Wikipedia mi
- Minangkabau Wikipedia min
- Macedonian Wikipedia mk
- Malayalam Wikipedia ml
- Mongolian Wikipedia mn
-
Mon Wikipedia mnwsee T308137#8690680 - Marathi Wikipedia mr
- Western Mari Wikipedia mrj
- Malay Wikipedia ms
- Maltese Wikipedia mt
- Mirandese Wikipedia mwl
-
Burmese Wikipedia mysee T308137#8690680 - Erzya Wikipedia myv
- Mazanderani Wikipedia mzn
-
Nauru Wikipedia nalocked wiki - Nāhuatl Wikipedia nah
- Neapolitan Wikipedia nap
- Low German Wikipedia nds
- Low Saxon Wikipedia nds-nl
- Nepali Wikipedia ne
- Newari Wikipedia new
- Norwegian Nynorsk Wikipedia nn
- Models verification
- Publish Datasets
- Populate the excluded section titles
- Deploy back-end
- Check how the model works on the wikis
- In Search, use hasrecommendation:link to find articles
- Test them on https://api.wikimedia.org/service/linkrecommendation/apidocs/#/default/get_v1_linkrecommendations__project___domain___page_title_
- Inform communities
- Deploy front-end
- Due Date
- Sep 6 2023, 4:00 PM
Event Timeline
Model evaluation has been completed and below are the backtesting results:
Precision@0.5 | Recall@0.5 | |
mhrwiki | 0.93 | 0.34 |
miwiki | 0.94 | 0.75 |
minwiki | 0.97 | 0.96 |
mkwiki | 0.87 | 0.57 |
mlwiki | 0.69 | 0.14 |
mnwiki | 0.72 | 0.30 |
mnwwiki | 0.00 | 0.00 |
mrwiki | 0.97 | 0.67 |
mrjwiki | 0.93 | 0.54 |
mswiki | 0.90 | 0.68 |
mtwiki | 0.75 | 0.35 |
mwlwiki | 0.84 | 0.56 |
mywiki | 0.63 | 0.06 |
myvwiki | 0.79 | 0.32 |
mznwiki | 0.93 | 0.71 |
nawiki | 0.95 | 0.86 |
nahwiki | 0.83 | 0.51 |
napwiki | 0.91 | 0.52 |
ndswiki | 0.89 | 0.44 |
nds_nlwiki | 0.82 | 0.55 |
newiki | 0.74 | 0.15 |
newwiki | 1.00 | 0.24 |
nnwiki | 0.86 | 0.64 |
CCing @MGerlach, in case he would like to add comments on the backtesting evaluation.
The conclusion on the backtesting results is that most of the languages look fine besides:
- mnwwiki's precision and recall are bad. Both are nil.
- mnwiki (0.72) and newiki (0.74) have a precision slightly lower than the recommended one (0.75).
- mlwiki's precision (0.69) and recall (0.14) are lower than the recommended one (0.75 and 0.2 respectively).
- mywiki has a low precision (0.63) and very low recall (0.06).
Talked to @MGerlach about these results and agreed mnwiki, newiki, and mlwiki should be deployed but mnwwiki and mywiki shouldn't.
Sgs changed the task status from Open to In Progress.Jul 5 2023, 2:07 PM
Sgs claimed this task.
Sgs triaged this task as Medium priority.
Change 935723 had a related patch set uploaded (by Sergio Gimeno; author: Sergio Gimeno):
[operations/mediawiki-config@master] GrowthExperiments: Enable backend of link recommendation 10, 11, 12th round wikis
I ran this script for adding the link-recommendation task type and populating the excluded sections entries:
for WIKI in mhrwiki miwiki minwiki mkwiki mlwiki mnwiki mrwiki mrjwiki mswiki mtwiki mwlwiki myvwiki mznwiki nawiki nahwiki napwiki ndswiki nds_nlwiki newiki newwiki nnwiki; do ORIGIN=`mwscript getConfiguration.php $WIKI --settings 'wgCanonicalServer' --format json | jq --raw-output '.wgCanonicalServer'` mwscript extensions/GrowthExperiments/maintenance/changeWikiConfig.php $WIKI \ --page MediaWiki:NewcomerTasks.json \ --create-only \ --json \ --summary "Growth features configuration boilerplate ([[phab:$PHAB]])" \ link-recommendation \ '{ "type": "link-recommendation", "group": "easy" }' jq "select(.wiki==\"$WIKI\" and .probability > 0.25) | .section" wiki_sections.jsonl \ | jq --slurp --compact-output "unique" \ | mwscript extensions/GrowthExperiments/maintenance/changeWikiConfig.php $WIKI \ --page MediaWiki:NewcomerTasks.json \ --json \ --summary "machine-generated configuration for excluding sections from link recommendations ([[phab:$PHAB]]), feel free to improve" \ link-recommendation.excludedSections \ "`cat`" echo "$ORIGIN/wiki/MediaWiki:NewcomerTasks.json" echo "$ORIGIN/w/index.php?title=MediaWiki:NewcomerTasks.json&diff=next" echo "Press <Enter> to continue" read # give time for manual verification done
Mentioned in SAL (#wikimedia-operations) [2023-07-11T13:04:58Z] <urbanecm@deploy1002> sgimeno and urbanecm: Backport for [[gerrit:935723|GrowthExperiments: Enable backend of link recommendation 10, 11, 12th round wikis (T308135 T308136 T308137)]] synced to the testservers: mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet
Status update, as per today all wikis have produced results except for nawiki.
For more context:
- mlwiki and mswiki were missing in wikis.txt file ( cc @kevinbazira ), I've added them and run the recommendations script on demand for these two and they have produced results.
- nawiki the recommendation script has failed to generate results without yielding any errors, but most probably because the wiki has been locked, see https://meta.wikimedia.org/wiki/Proposals_for_closing_projects/Closure_of_Nauruan_Wikipedia
I think we can proceed with the rollout of this round except for nawiki. cc @Trizek-WMF.
Thank you for conducting the tests. :)
We won't deploy at na.wp. It was locked after I created the lists.
I set the date to deploy on Sept 6th.
Mentioned in SAL (#wikimedia-operations) [2023-09-06T20:04:40Z] <taavi@deploy1002> taavi and sgimeno: Backport for [[gerrit:948144|GrowthExperiments: enable add a link in 12 and 13th round of wikis (T308137 T308138)]] synced to the testservers mwdebug1001.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet, and mw-debug kubernetes deployment (accessible via k8s-experimental XWD option)
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits