> For the complete documentation index, see [llms.txt](https://langtech-bsc.gitbook.io/alia-kit/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://langtech-bsc.gitbook.io/alia-kit/datasets/datos-y-herramientas-para-modelos-de-texto/datasets-de-pre-entrenamiento-alia-40b.md).

# Datasets de pre entrenamiento ALIA 40B

Relación de datasets con referencias en el marco del pre entrenamiento del modelo ALIA 40B.

Los corpus de entrenamiento se listan a continuación:

<table data-header-hidden><thead><tr><th valign="bottom"></th><th valign="bottom"></th><th valign="bottom"></th></tr></thead><tbody><tr><td valign="bottom">Corpus</td><td valign="bottom">Lenguas</td><td valign="bottom">Link</td></tr><tr><td valign="bottom">Colossal OSCAR 1.0</td><td valign="bottom">bg, ca, cs, cy, da, de, el, en, es, et, eu, fi, fr, ga, gl, hr, hu, it, lt, lv, mt, nl, nn, no, oc, pl, pt, ro, ru, sh, sk, sl, sr, sv, uk</td><td valign="bottom"><a href="https://huggingface.co/datasets/oscar-corpus/colossal-oscar-1.0">https://huggingface.co/datasets/oscar-corpus/colossal-oscar-1.0</a></td></tr><tr><td valign="bottom">Aya Dataset (w/o Evaluation Suite)</td><td valign="bottom">eu, hr, nl, fi, ka, hu, lt, nn, ro, sk, lv, cy, bg, cs, en, fr, de, ga, mt, pl, ru, sl, sv, ca, da, et, gl, el, it, no, pt, sr, es, uk</td><td valign="bottom"><a href="https://huggingface.co/datasets/CohereForAI/aya_dataset">https://huggingface.co/datasets/CohereForAI/aya_dataset</a></td></tr><tr><td valign="bottom">Wikimedia dumps</td><td valign="bottom">bg, ca, cs, da, de, el, en, es, et, eu, fi, fr, ga, gl, hr, hu, it, lt, lv, mt, nl, nn, no, pl, pt, ro, sh, sk, sl, sr, uk</td><td valign="bottom"><a href="https://dumps.wikimedia.org/">https://dumps.wikimedia.org/</a></td></tr><tr><td valign="bottom">OpenSubtitles v2016</td><td valign="bottom">bg, ca, cs, da, de, el, en, es, et, eu, fi, fr, gl, hr, it, lt, lv, nl, no, pl, pt, ro, sk, sl, sr, sv, uk</td><td valign="bottom"><a href="https://huggingface.co/datasets/Helsinki-NLP/open_subtitles">https://huggingface.co/datasets/Helsinki-NLP/open_subtitles</a></td></tr><tr><td valign="bottom">EurLEX-Resources</td><td valign="bottom">bg, cs, da, de, el, en, es, et, fi, fr, ga, hr, hu, it, lt, lv, mt, nl, pl, pt, ro, sk, sl, sv</td><td valign="bottom"><a href="https://huggingface.co/datasets/joelniklaus/eurlex_resources">https://huggingface.co/datasets/joelniklaus/eurlex_resources</a></td></tr><tr><td valign="bottom">MC4-Legal</td><td valign="bottom">bg, cs, da, de, el, en, es, et, fi, fr, ga, hu, it, lt, lv, mt, nl, pl, pt, ro, sk, sl, sv</td><td valign="bottom"><a href="https://huggingface.co/datasets/joelniklaus/legal-mc4">https://huggingface.co/datasets/joelniklaus/legal-mc4</a></td></tr><tr><td valign="bottom">Parlamint</td><td valign="bottom">at, bg, cz, dk, ee, es, es-ga, fi, fr, gb, gr, hr, hu, it, lv, nl, no, pl, pt, rs, se, si</td><td valign="bottom"><a href="https://clarin-eric.github.io/ParlaMint/">https://clarin-eric.github.io/ParlaMint/</a></td></tr><tr><td valign="bottom">MaCoCu</td><td valign="bottom">bg, ca, el, hr, mt, sl, sr, uk</td><td valign="bottom"><a href="https://macocu.eu/">https://macocu.eu/</a></td></tr><tr><td valign="bottom">CURLICAT</td><td valign="bottom">bg, hr, hu, pl, ro, sk, sl</td><td valign="bottom"><a href="https://curlicat-project.eu/">https://curlicat-project.eu/</a></td></tr><tr><td valign="bottom">Norwegian Colossal Corpus (NCC)</td><td valign="bottom">nn, no</td><td valign="bottom"><a href="https://github.com/NbAiLab/notram/blob/master/guides/corpus_description.md">https://github.com/NbAiLab/notram/blob/master/guides/corpus_description.md</a></td></tr><tr><td valign="bottom">Academic Slovene KAS 2.0</td><td valign="bottom">sl</td><td valign="bottom"><a href="https://www.clarin.si/repository/xmlui/handle/11356/1448">https://www.clarin.si/repository/xmlui/handle/11356/1448</a></td></tr><tr><td valign="bottom">BIGPATENT</td><td valign="bottom">en</td><td valign="bottom"><a href="https://huggingface.co/datasets/NortheasternUniversity/big_patent">https://huggingface.co/datasets/NortheasternUniversity/big_patent</a></td></tr><tr><td valign="bottom">Biomedical-ES</td><td valign="bottom">es</td><td valign="bottom"><a href="https://zenodo.org/records/4561971">https://zenodo.org/records/4561971</a></td></tr><tr><td valign="bottom">Brazilian Portuguese Web as Corpus (BrWaC)</td><td valign="bottom">pt</td><td valign="bottom"><a href="https://huggingface.co/datasets/dominguesm/brwac">https://huggingface.co/datasets/dominguesm/brwac</a></td></tr><tr><td valign="bottom">Bulgarian National Corpus (BulNC)</td><td valign="bottom">bg</td><td valign="bottom"><a href="http://old.dcl.bas.bg/dataset/BulNC.7z">http://old.dcl.bas.bg/dataset/BulNC.7z</a></td></tr><tr><td valign="bottom">CaBeRnet</td><td valign="bottom">fr</td><td valign="bottom"><a href="https://aclanthology.org/2020.cmlc-1.3/">https://aclanthology.org/2020.cmlc-1.3/</a></td></tr><tr><td valign="bottom">CATalog 1.0</td><td valign="bottom">ca</td><td valign="bottom"><a href="https://huggingface.co/datasets/projecte-aina/CATalog">https://huggingface.co/datasets/projecte-aina/CATalog</a></td></tr><tr><td valign="bottom">CorpusNÓS</td><td valign="bottom">gl</td><td valign="bottom"><a href="https://zenodo.org/records/11655219">https://zenodo.org/records/11655219</a></td></tr><tr><td valign="bottom">Croatian Web as Corpus 2.1 (hrWaC)</td><td valign="bottom">hr</td><td valign="bottom"><a href="https://clarin.si/repository/xmlui/handle/11356/1064">https://clarin.si/repository/xmlui/handle/11356/1064</a></td></tr><tr><td valign="bottom">DaNewsroom</td><td valign="bottom">da</td><td valign="bottom"><a href="https://github.com/danielvarab/da-newsroom">https://github.com/danielvarab/da-newsroom</a></td></tr><tr><td valign="bottom">Danish GigaWord</td><td valign="bottom">da</td><td valign="bottom"><a href="https://huggingface.co/datasets/danish-foundation-models/danish-gigaword">https://huggingface.co/datasets/danish-foundation-models/danish-gigaword</a></td></tr><tr><td valign="bottom">Dolmino-mix-1124 (subset without synthetically generated data and privative licenses)</td><td valign="bottom">en</td><td valign="bottom"><a href="https://huggingface.co/datasets/allenai/dolmino-mix-1124">https://huggingface.co/datasets/allenai/dolmino-mix-1124</a></td></tr><tr><td valign="bottom">DK-CLARIN Reference Corpus of General Danish</td><td valign="bottom">da</td><td valign="bottom"><a href="https://korpus.dsl.dk/clarin/">https://korpus.dsl.dk/clarin/</a></td></tr><tr><td valign="bottom">Estonian National Corpus 2021 (ENC)</td><td valign="bottom">et</td><td valign="bottom"><a href="https://metashare.ut.ee/repository/search/?q=estonian%20national%20corpus">https://metashare.ut.ee/repository/search/?q=estonian%20national%20corpus</a></td></tr><tr><td valign="bottom">Estonian Reference Corpus (ERC)</td><td valign="bottom">et</td><td valign="bottom"><a href="https://lindat.mff.cuni.cz/repository/xmlui/handle/11372/LRT-1068">https://lindat.mff.cuni.cz/repository/xmlui/handle/11372/LRT-1068</a></td></tr><tr><td valign="bottom">EusCrawl (w/o Wikipedia or NC-licenses)</td><td valign="bottom">eu</td><td valign="bottom"><a href="https://huggingface.co/datasets/HiTZ/euscrawl">https://huggingface.co/datasets/HiTZ/euscrawl</a></td></tr><tr><td valign="bottom">FineWeb-Edu (350BT subset)</td><td valign="bottom">en</td><td valign="bottom"><a href="https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu">https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu</a></td></tr><tr><td valign="bottom">Fineweb2 (ad hoc subset of 178BT)</td><td valign="bottom">ar, as, bg, ca, cs, cy, da, de, el, es, et, eu, fi, fr, ga, gl, hr, hu, it, lt, lv, mt, nl, nn, no, oc, pl, pt, ro, ru, sk, sl, sr, sv, uk</td><td valign="bottom"><a href="https://huggingface.co/datasets/HuggingFaceFW/fineweb-2">https://huggingface.co/datasets/HuggingFaceFW/fineweb-2</a></td></tr><tr><td valign="bottom">French Public Domain Books (French-PD)</td><td valign="bottom">fr</td><td valign="bottom"><a href="https://huggingface.co/datasets/PleIAs/French-PD-Books">https://huggingface.co/datasets/PleIAs/French-PD-Books</a></td></tr><tr><td valign="bottom">French Public Domain Newspapers (French-PD)</td><td valign="bottom">fr</td><td valign="bottom"><a href="https://huggingface.co/datasets/PleIAs/French-PD-Newspapers">https://huggingface.co/datasets/PleIAs/French-PD-Newspapers</a></td></tr><tr><td valign="bottom">German Web as Corpus (DeWaC)</td><td valign="bottom">de</td><td valign="bottom"><a href="https://wacky.sslmit.unibo.it/doku.php?id=seed_urls">https://wacky.sslmit.unibo.it/doku.php?id=seed_urls</a></td></tr><tr><td valign="bottom">Greek Legal Code (GLC)</td><td valign="bottom">el</td><td valign="bottom"><a href="https://huggingface.co/datasets/AI-team-UoA/greek_legal_code">https://huggingface.co/datasets/AI-team-UoA/greek_legal_code</a></td></tr><tr><td valign="bottom">Greek Web Corpus (GWC)</td><td valign="bottom">el</td><td valign="bottom"><a href="http://nlp.polytechnique.fr/resources-greek">http://nlp.polytechnique.fr/resources-greek</a></td></tr><tr><td valign="bottom">HPLT v1 - Spanish</td><td valign="bottom">es</td><td valign="bottom"><a href="https://hplt-project.org/datasets/v1">https://hplt-project.org/datasets/v1</a></td></tr><tr><td valign="bottom">HPLT v1.1 - Spanish</td><td valign="bottom">es</td><td valign="bottom"><a href="https://hplt-project.org/datasets/v1.1">https://hplt-project.org/datasets/v1.1</a></td></tr><tr><td valign="bottom">Irish Universal Dependencies (Ga-UD)</td><td valign="bottom">ga</td><td valign="bottom"><a href="https://universaldependencies.org/ga/">https://universaldependencies.org/ga/</a></td></tr><tr><td valign="bottom">Italian Web as Corpus (ItWaC)</td><td valign="bottom">it</td><td valign="bottom"><a href="https://wacky.sslmit.unibo.it/doku.php?id=seed_urls">https://wacky.sslmit.unibo.it/doku.php?id=seed_urls</a></td></tr><tr><td valign="bottom">Korpus Malti</td><td valign="bottom">mt</td><td valign="bottom"><a href="https://huggingface.co/datasets/MLRS/korpus_malti">https://huggingface.co/datasets/MLRS/korpus_malti</a></td></tr><tr><td valign="bottom">Korpus slovenských právnych predpisov v1.9 (SK-Laws)</td><td valign="bottom">sk</td><td valign="bottom"><a href="https://www.juls.savba.sk/data.html">https://www.juls.savba.sk/data.html</a></td></tr><tr><td valign="bottom">Latxa Corpus v1.1 (GAITU)</td><td valign="bottom">eu</td><td valign="bottom"><a href="https://huggingface.co/datasets/HiTZ/latxa-corpus-v1.1">https://huggingface.co/datasets/HiTZ/latxa-corpus-v1.1</a></td></tr><tr><td valign="bottom">Laws and legal acts of Ukraine (UK-Laws)</td><td valign="bottom">uk</td><td valign="bottom"><a href="https://lang.org.ua/en/corpora/#anchor7">https://lang.org.ua/en/corpora/#anchor7</a></td></tr><tr><td valign="bottom">Legal-ES</td><td valign="bottom">es</td><td valign="bottom"><a href="https://aclanthology.org/2020.lt4gov-1.6/">https://aclanthology.org/2020.lt4gov-1.6/</a></td></tr><tr><td valign="bottom">MARCELL Romanian legislative subcorpus v2</td><td valign="bottom">ro</td><td valign="bottom"><a href="https://elrc-share.eu/repository/browse/marcell-romanian-legislative-subcorpus-v2/2da548428b9d11eb9c1a00155d026706ce94a6b59ffc4b0e9fb5cd9cebe6889e/">https://elrc-share.eu/repository/browse/marcell-romanian-legislative-subcorpus-v2/2da548428b9d11eb9c1a00155d026706ce94a6b59ffc4b0e9fb5cd9cebe6889e/</a></td></tr><tr><td valign="bottom">Math AMPS</td><td valign="bottom">en</td><td valign="bottom"><a href="https://github.com/hendrycks/math">https://github.com/hendrycks/math</a></td></tr><tr><td valign="bottom">NKPJ National Corpus of Polish v1.2 (NKPJ)</td><td valign="bottom">pl</td><td valign="bottom"><a href="https://nkjp.pl/index.php?page=0&#x26;lang=1">https://nkjp.pl/index.php?page=0&#x26;lang=1</a></td></tr><tr><td valign="bottom">Occitan Corpus (IEA-AALO)</td><td valign="bottom">oc</td><td valign="bottom"><p>Datos descargados de la web del i<em>nstitut</em> a través de acuerdo, no publicados.</p><p><a href="https://www.institutestudisaranesi.cat/">https://www.institutestudisaranesi.cat/</a></p></td></tr><tr><td valign="bottom">Open Legal Data - German court decisions and laws</td><td valign="bottom">de</td><td valign="bottom"><p><a href="https://openlegaldata.io/">https://openlegaldata.io/</a></p><p>enlace de descarga desactualizado</p></td></tr><tr><td valign="bottom">ParlamentoPT</td><td valign="bottom">pt</td><td valign="bottom"><a href="https://huggingface.co/datasets/PORTULAN/parlamento-pt">https://huggingface.co/datasets/PORTULAN/parlamento-pt</a></td></tr><tr><td valign="bottom">peS2o</td><td valign="bottom">en</td><td valign="bottom"><a href="https://huggingface.co/datasets/allenai/peS2o">https://huggingface.co/datasets/allenai/peS2o</a></td></tr><tr><td valign="bottom">PG-19</td><td valign="bottom">en</td><td valign="bottom"><a href="https://huggingface.co/datasets/deepmind/pg19">https://huggingface.co/datasets/deepmind/pg19</a></td></tr><tr><td valign="bottom">Pile of Law (selected subsets)</td><td valign="bottom">en</td><td valign="bottom"><a href="https://huggingface.co/datasets/pile-of-law/pile-of-law">https://huggingface.co/datasets/pile-of-law/pile-of-law</a></td></tr><tr><td valign="bottom">Polish Parliamentary Corpus (PPC)</td><td valign="bottom">pl</td><td valign="bottom"><a href="https://clip.ipipan.waw.pl/PPC">https://clip.ipipan.waw.pl/PPC</a></td></tr><tr><td valign="bottom">Proof Pile</td><td valign="bottom">en</td><td valign="bottom"><a href="https://huggingface.co/datasets/hoskinson-center/proof-pile">https://huggingface.co/datasets/hoskinson-center/proof-pile</a></td></tr><tr><td valign="bottom">RedPajama-Data T1 (StackExchange subset)</td><td valign="bottom">en</td><td valign="bottom"><a href="https://huggingface.co/datasets/togethercomputer/RedPajama-Data-1T">https://huggingface.co/datasets/togethercomputer/RedPajama-Data-1T</a></td></tr><tr><td valign="bottom">Scientific-ES</td><td valign="bottom">es</td><td valign="bottom">Recopilación de repositorios en abierto (<a href="https://dialnet.unirioja.es/">https://dialnet.unirioja.es/</a>, <a href="https://scielo.isciii.es/scielo.php">https://scielo.isciii.es/scielo.php</a>, <a href="https://revistas.csic.es/">https://revistas.csic.es/</a>, <a href="https://www.tesisenred.net/">https://www.tesisenred.net</a>/, <a href="https://docta.ucm.es/home">https://docta.ucm.es/home</a>)</td></tr><tr><td valign="bottom">SK Court Decisions v2.0 (OD-Justice)</td><td valign="bottom">sk</td><td valign="bottom"><a href="https://www.juls.savba.sk/data/od-justice/od-justice-2.0.ver.xz">https://www.juls.savba.sk/data/od-justice/od-justice-2.0.ver.xz</a></td></tr><tr><td valign="bottom">Slovene Web as Corpus (slWaC)</td><td valign="bottom">sl</td><td valign="bottom"><a href="https://www.sketchengine.eu/slwac-slovenian-corpus-from-the-web/">https://www.sketchengine.eu/slwac-slovenian-corpus-from-the-web/</a></td></tr><tr><td valign="bottom">SoNaR Corpus NC 1.2</td><td valign="bottom">nl</td><td valign="bottom"><a href="https://elrc-share.eu/repository/browse/sonar-corpus/9735a54f1f9111e7bfe700155d020502b917ac3b8c8844e19665914d110e94d1/">https://elrc-share.eu/repository/browse/sonar-corpus/9735a54f1f9111e7bfe700155d020502b917ac3b8c8844e19665914d110e94d1/</a></td></tr><tr><td valign="bottom">Spanish Legal Domain Corpora (Spanish-Legal)</td><td valign="bottom">es</td><td valign="bottom"><a href="https://zenodo.org/records/5495529">https://zenodo.org/records/5495529</a></td></tr><tr><td valign="bottom">SrpKorSubset: news, legal, academic, conversation, lit- erary (SrpKor)</td><td valign="bottom">sr</td><td valign="bottom"><a href="http://metashare.elda.org/repository/browse/corpus-of-contemporary-serbian/00cc41168bdf11e29c9e0015171445924cdac8693bf840f780418187133495b8/">http://metashare.elda.org/repository/browse/corpus-of-contemporary-serbian/00cc41168bdf11e29c9e0015171445924cdac8693bf840f780418187133495b8/</a></td></tr><tr><td valign="bottom">Starcoder</td><td valign="bottom">code</td><td valign="bottom"><a href="https://huggingface.co/datasets/bigcode/starcoderdata">https://huggingface.co/datasets/bigcode/starcoderdata</a></td></tr><tr><td valign="bottom">State-related content from the Latvian Web (State-Latvian-Web)</td><td valign="bottom">lv</td><td valign="bottom"><a href="https://catalog.elra.info/en-us/repository/browse/ELRA-W0169/">https://catalog.elra.info/en-us/repository/browse/ELRA-W0169/</a></td></tr><tr><td valign="bottom">SYN v9: large corpus of written Czech</td><td valign="bottom">cs</td><td valign="bottom"><a href="https://lindat.mff.cuni.cz/repository/xmlui/handle/11234/1-4635">https://lindat.mff.cuni.cz/repository/xmlui/handle/11234/1-4635</a></td></tr><tr><td valign="bottom">Tagesschau Archive Article</td><td valign="bottom">de</td><td valign="bottom"><a href="https://huggingface.co/datasets/bjoernp/tagesschau-2018-2023">https://huggingface.co/datasets/bjoernp/tagesschau-2018-2023</a></td></tr><tr><td valign="bottom">The Danish Parliament Corpus 2009 - 2017, v1</td><td valign="bottom">da</td><td valign="bottom"><a href="https://repository.clarin.dk/repository/xmlui/handle/20.500.12115/8">https://repository.clarin.dk/repository/xmlui/handle/20.500.12115/8</a></td></tr><tr><td valign="bottom">The Gaois bilingual corpus of English-Irish legislation (Ga-Legislation)</td><td valign="bottom">ga</td><td valign="bottom"><a href="https://portulanclarin.net/repository/browse/the-gaois-bilingual-corpus-of-english-irish-legislation-processed/daeac17c9e3511ea9b7f02420a000407b83de243dc0b469aab41084386c5b80f/">https://portulanclarin.net/repository/browse/the-gaois-bilingual-corpus-of-english-irish-legislation-processed/daeac17c9e3511ea9b7f02420a000407b83de243dc0b469aab41084386c5b80f/</a></td></tr><tr><td valign="bottom">The Pile (PhilPapers)</td><td valign="bottom">en</td><td valign="bottom"><a href="https://github.com/thoppe/The-Pile-PhilPapers">https://github.com/thoppe/The-Pile-PhilPapers</a></td></tr><tr><td valign="bottom">The Swedish Culturomics Gigaword Corpus (Swedish- Gigaword)</td><td valign="bottom">sv</td><td valign="bottom"><a href="https://spraakbanken.gu.se/en/resources/gigaword">https://spraakbanken.gu.se/en/resources/gigaword</a></td></tr><tr><td valign="bottom">Welsh-GOV</td><td valign="bottom">cy</td><td valign="bottom"><a href="https://www.llyw.cymru/">Crawling de aqui: https://www.llyw.cymru/</a></td></tr><tr><td valign="bottom">Yle Finnish News Archive (Yle-News)</td><td valign="bottom"> </td><td valign="bottom"><a href="https://www.kielipankki.fi/download/YLE/fi/2019-2020-src/">https://www.kielipankki.fi/download/YLE/fi/2019-2020-src/</a></td></tr></tbody></table>


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://langtech-bsc.gitbook.io/alia-kit/datasets/datos-y-herramientas-para-modelos-de-texto/datasets-de-pre-entrenamiento-alia-40b.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.