# bert-mlm-base

**Repository Path**: modelee/bert-mlm-base

## Basic Information

- **Project Name**: bert-mlm-base
- **Description**: No description available
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: main
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 3
- **Forks**: 1
- **Created**: 2023-05-23
- **Last Updated**: 2025-05-22

## Categories & Tags

**Categories**: llm

**Tags**: None

## README

---
tags: 
- bert
license: cc-by-4.0
---
## bert-mlm-base
A BERT base Language Model with an **MLM** pre-training objective. For more details about the pre-training objective and the pre-training hyperparameters, please refer to [How does the pre-training objective affect what large language models learn about linguistic properties?](https://aclanthology.org/2022.acl-short.16/)

## License
CC BY 4.0

## Citation
If you use this model, please cite the following paper:
```
@inproceedings{alajrami2022does,
  title={How does the pre-training objective affect what large language models learn about linguistic properties?},
  author={Alajrami, Ahmed and Aletras, Nikolaos},
  booktitle={Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)},
  pages={131--147},
  year={2022}
}
```