⚓ T308137 Deploy "add a link" to 12th round of wikis


Article Images

Deploy "add a link" to 12th round of wikis

  • Training models
    • Eastern Mari Wikipedia mhr
    • Maori Wikipedia mi
    • Minangkabau Wikipedia min
    • Macedonian Wikipedia mk
    • Malayalam Wikipedia ml
    • Mongolian Wikipedia mn
    • Mon Wikipedia mnw see T308137#8690680
    • Marathi Wikipedia mr
    • Western Mari Wikipedia mrj
    • Malay Wikipedia ms
    • Maltese Wikipedia mt
    • Mirandese Wikipedia mwl
    • Burmese Wikipedia my see T308137#8690680
    • Erzya Wikipedia myv
    • Mazanderani Wikipedia mzn
    • Nauru Wikipedia na locked wiki
    • Nāhuatl Wikipedia nah
    • Neapolitan Wikipedia nap
    • Low German Wikipedia nds
    • Low Saxon Wikipedia nds-nl
    • Nepali Wikipedia ne
    • Newari Wikipedia new
    • Norwegian Nynorsk Wikipedia nn
  • Models verification
  • Publish Datasets
  • Populate the excluded section titles
  • Deploy back-end
  • Check how the model works on the wikis
  • In Search, use hasrecommendation:link to find articles
  • Test them on https://api.wikimedia.org/service/linkrecommendation/apidocs/#/default/get_v1_linkrecommendations__project___domain___page_title_
  • Inform communities
  • Deploy front-end
Due Date
Sep 6 2023, 4:00 PM

Event Timeline

23/23 models were trained successfully in the 12th round of wikis.

Model evaluation has been completed and below are the backtesting results:

Precision@0.5Recall@0.5
mhrwiki0.930.34
miwiki0.940.75
minwiki0.970.96
mkwiki0.870.57
mlwiki0.690.14
mnwiki0.720.30
mnwwiki0.000.00
mrwiki0.970.67
mrjwiki0.930.54
mswiki0.900.68
mtwiki0.750.35
mwlwiki0.840.56
mywiki0.630.06
myvwiki0.790.32
mznwiki0.930.71
nawiki0.950.86
nahwiki0.830.51
napwiki0.910.52
ndswiki0.890.44
nds_nlwiki0.820.55
newiki0.740.15
newwiki1.000.24
nnwiki0.860.64

CCing @MGerlach, in case he would like to add comments on the backtesting evaluation.

The conclusion on the backtesting results is that most of the languages look fine besides:

  • mnwwiki's precision and recall are bad. Both are nil.
  • mnwiki (0.72) and newiki (0.74) have a precision slightly lower than the recommended one (0.75).
  • mlwiki's precision (0.69) and recall (0.14) are lower than the recommended one (0.75 and 0.2 respectively).
  • mywiki has a low precision (0.63) and very low recall (0.06).

Talked to @MGerlach about these results and agreed mnwiki, newiki, and mlwiki should be deployed but mnwwiki and mywiki shouldn't.

Sgs changed the task status from Open to In Progress.Jul 5 2023, 2:07 PM

Sgs claimed this task.

Sgs triaged this task as Medium priority.

Change 935723 had a related patch set uploaded (by Sergio Gimeno; author: Sergio Gimeno):

[operations/mediawiki-config@master] GrowthExperiments: Enable backend of link recommendation 10, 11, 12th round wikis

https://gerrit.wikimedia.org/r/935723

I ran this script for adding the link-recommendation task type and populating the excluded sections entries:

for WIKI in mhrwiki miwiki minwiki mkwiki mlwiki mnwiki mrwiki mrjwiki mswiki mtwiki mwlwiki myvwiki mznwiki nawiki nahwiki napwiki ndswiki nds_nlwiki newiki newwiki nnwiki; do
    ORIGIN=`mwscript getConfiguration.php $WIKI --settings 'wgCanonicalServer' --format json | jq --raw-output '.wgCanonicalServer'`
    mwscript extensions/GrowthExperiments/maintenance/changeWikiConfig.php $WIKI \
            --page MediaWiki:NewcomerTasks.json \
            --create-only \
            --json \
            --summary "Growth features configuration boilerplate ([[phab:$PHAB]])" \
            link-recommendation \
            '{ "type": "link-recommendation", "group": "easy" }'
    jq "select(.wiki==\"$WIKI\" and .probability > 0.25) | .section" wiki_sections.jsonl \
        | jq --slurp --compact-output "unique" \
        | mwscript extensions/GrowthExperiments/maintenance/changeWikiConfig.php $WIKI \
            --page MediaWiki:NewcomerTasks.json \
            --json \
            --summary "machine-generated configuration for excluding sections from link recommendations ([[phab:$PHAB]]), feel free to improve" \
            link-recommendation.excludedSections \
            "`cat`"
    echo "$ORIGIN/wiki/MediaWiki:NewcomerTasks.json"
    echo "$ORIGIN/w/index.php?title=MediaWiki:NewcomerTasks.json&diff=next"
    echo "Press <Enter> to continue"
    read # give time for manual verification
done

Mentioned in SAL (#wikimedia-operations) [2023-07-11T13:04:58Z] <urbanecm@deploy1002> sgimeno and urbanecm: Backport for [[gerrit:935723|GrowthExperiments: Enable backend of link recommendation 10, 11, 12th round wikis (T308135 T308136 T308137)]] synced to the testservers: mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet

Status update, as per today all wikis have produced results except for nawiki.

For more context:

I think we can proceed with the rollout of this round except for nawiki. cc @Trizek-WMF.

Thank you for conducting the tests. :)

We won't deploy at na.wp. It was locked after I created the lists.

I set the date to deploy on Sept 6th.

Mentioned in SAL (#wikimedia-operations) [2023-09-06T20:04:40Z] <taavi@deploy1002> taavi and sgimeno: Backport for [[gerrit:948144|GrowthExperiments: enable add a link in 12 and 13th round of wikis (T308137 T308138)]] synced to the testservers mwdebug1001.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet, and mw-debug kubernetes deployment (accessible via k8s-experimental XWD option)

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits