| 
| 
| 
| 
|
+|:------------------------------------------------------------------:|:---------------------------------------------------------------:|:--------------------------------------------------------------:|:-------------------------------------------------------:|:-----------------------------------------------------------:|:------------------------------------------------------:|
+
+The contributors can be found in [contributors list](./Acknowledge.md)
+
+## License
+This project is licensed under [The MIT License](https://opensource.org/licenses/MIT). FunASR also contains various third-party components and some code modified from other repos under other open source licenses.
+The use of pretraining model is subject to [model license](./MODEL_LICENSE)
+
+
+## Citations
+``` bibtex
+@inproceedings{gao2023funasr,
+ author={Zhifu Gao and Zerui Li and Jiaming Wang and Haoneng Luo and Xian Shi and Mengzhe Chen and Yabin Li and Lingyun Zuo and Zhihao Du and Zhangyu Xiao and Shiliang Zhang},
+ title={FunASR: A Fundamental End-to-End Speech Recognition Toolkit},
+ year={2023},
+ booktitle={INTERSPEECH},
+}
+@inproceedings{An2023bat,
+ author={Keyu An and Xian Shi and Shiliang Zhang},
+ title={BAT: Boundary aware transducer for memory-efficient and low-latency ASR},
+ year={2023},
+ booktitle={INTERSPEECH},
+}
+@inproceedings{gao22b_interspeech,
+ author={Zhifu Gao and ShiLiang Zhang and Ian McLoughlin and Zhijie Yan},
+ title={Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognition},
+ year=2022,
+ booktitle={Proc. Interspeech 2022},
+ pages={2063--2067},
+ doi={10.21437/Interspeech.2022-9996}
+}
+@inproceedings{shi2023seaco,
+ author={Xian Shi and Yexin Yang and Zerui Li and Yanni Chen and Zhifu Gao and Shiliang Zhang},
+ title={SeACo-Paraformer: A Non-Autoregressive ASR System with Flexible and Effective Hotword Customization Ability},
+ year={2023},
+ booktitle={ICASSP2024}
+}
+```
diff --git a/README_zh.md b/README_zh.md
new file mode 100644
index 0000000..d3c9ff6
--- /dev/null
+++ b/README_zh.md
@@ -0,0 +1,310 @@
+[//]: # (
| | | | |
+|:------------------------------------------------------------------:|:---------------------------------------------------------------:|:--------------------------------------------------------------:|:-------------------------------------------------------:|:-----------------------------------------------------------:|:------------------------------------------------------:|
+
+贡献者名单请参考（[致谢名单](./Acknowledge.md)）
+
+
+## 许可协议
+项目遵循[The MIT License](https://opensource.org/licenses/MIT)开源协议，模型许可协议请参考（[模型协议](./MODEL_LICENSE)）
+
+
+## 论文引用
+
+``` bibtex
+@inproceedings{gao2023funasr,
+ author={Zhifu Gao and Zerui Li and Jiaming Wang and Haoneng Luo and Xian Shi and Mengzhe Chen and Yabin Li and Lingyun Zuo and Zhihao Du and Zhangyu Xiao and Shiliang Zhang},
+ title={FunASR: A Fundamental End-to-End Speech Recognition Toolkit},
+ year={2023},
+ booktitle={INTERSPEECH},
+}
+@inproceedings{An2023bat,
+ author={Keyu An and Xian Shi and Shiliang Zhang},
+ title={BAT: Boundary aware transducer for memory-efficient and low-latency ASR},
+ year={2023},
+ booktitle={INTERSPEECH},
+}
+@inproceedings{gao22b_interspeech,
+ author={Zhifu Gao and ShiLiang Zhang and Ian McLoughlin and Zhijie Yan},
+ title={{Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognition}},
+ year=2022,
+ booktitle={Proc. Interspeech 2022},
+ pages={2063--2067},
+ doi={10.21437/Interspeech.2022-9996}
+}
+@article{shi2023seaco,
+ author={Xian Shi and Yexin Yang and Zerui Li and Yanni Chen and Zhifu Gao and Shiliang Zhang},
+ title={{SeACo-Paraformer: A Non-Autoregressive ASR System with Flexible and Effective Hotword Customization Ability}},
+ year=2023,
+ journal={arXiv preprint arXiv:2308.03266(accepted by ICASSP2024)},
+}
+```
diff --git a/benchmarks/benchmark_pipeline_cer.md b/benchmarks/benchmark_pipeline_cer.md
new file mode 100644
index 0000000..33d8571
--- /dev/null
+++ b/benchmarks/benchmark_pipeline_cer.md
@@ -0,0 +1,216 @@
+# Leaderboard IO
+
+
+## Configuration
+### Data set:
+[Aishell1](https://www.openslr.org/33/): dev, test
+
+[Aishell2](https://www.aishelltech.com/aishell_2): dev_ios, test_ios, test_android, test_mic
+
+[WenetSpeech](https://github.com/wenet-e2e/WenetSpeech): dev, test_meeting, test_net
+
+
+### Tools
+#### [Install Requirements](https://alibaba-damo-academy.github.io/FunASR/en/installation/installation.html#installation)
+
+Install ModelScope and FunASR from pip
+```shell
+pip install -U modelscope funasr
+# For the users in China, you could install with the command:
+#pip install -U funasr -i https://mirror.sjtu.edu.cn/pypi/web/simple
+```
+
+Or install FunASR from source code
+```shell
+git clone https://github.com/alibaba/FunASR.git && cd FunASR
+pip install -e ./
+# For the users in China, you could install with the command:
+# pip install -e ./ -i https://mirror.sjtu.edu.cn/pypi/web/simple
+```
+
+
+#### Recipe
+
+
+##### [Test CER](https://alibaba-damo-academy.github.io/FunASR/en/modelscope_pipeline/asr_pipeline.html#inference-with-multi-thread-cpus-or-multi-gpus)
+set the `model`, `data_dir` and `output_dir` in `infer.sh`.
+```shell
+cd egs_modelscope/asr/TEMPLATE
+bash infer.sh
+```
+
+## Benchmark CER
+
+
+### Chinese Dataset
+
+
+| Model | +Offline/Online | +Aishell1 | +Aishell2 | +WenetSpeech | +||||||
| + | + | dev | +test | +dev_ios | +test_ios | +test_android | +test_mic | +dev | +test_meeting | +test_net | +
| Paraformer-large | +Offline | +1.76 | +1.94 | +2.79 | +2.84 | +3.08 | +3.03 | +3.43 | +7.01 | +6.66 | +
| Paraformer-large-long | +Offline | +1.80 | +2.10 | +2.78 | +2.87 | +3.12 | +3.11 | +3.44 | +13.28 | +7.08 | +
| Paraformer-large-contextual | +Offline | +1.76 | +2.02 | +2.73 | +2.85 | +2.98 | +2.95 | +3.42 | +7.16 | +6.72 | +
| Paraformer-large-online | +Online | +2.37 | +3.34 | +4.04 | +3.86 | +4.38 | +4.21 | +4.55 | +10.64 | +7.78 | +
| Paraformer | +Offline | +3.24 | +3.69 | +4.58 | +4.63 | +4.83 | +4.71 | +4.19 | +8.32 | +9.19 | +
| UniASR | +Online | +3.34 | +3.99 | +4.62 | +4.52 | +4.77 | +4.73 | +4.51 | +10.63 | +9.70 | +
| UniASR-large | +Offline | +2.93 | +3.48 | +3.95 | +3.87 | +4.11 | +4.11 | +4.16 | +10.09 | +8.69 | +
| Paraformer-aishell | +Offline | +4.88 | +5.43 | +- | +- | +- | +- | +- | +- | +- | +
| ParaformerBert-aishell | +Offline | +6.14 | +7.01 | +- | +- | +- | +- | +- | +- | +- | +
| Paraformer-aishell2 | +Offline | +- | +- | +5.82 | +6.30 | +6.60 | +5.83 | +- | +- | +- | +
| ParaformerBert-aishell2 | +Offline | +- | +- | +4.95 | +5.45 | +5.59 | +5.83 | +- | +- | +- | +
+ 
+ 
+ 
+
+
+
+
+
+***Kong Aik Lee, Senior Scientist at Institute for Infocomm Research, A\*Star, Singapore***
+
+Email: [kongaik.lee@ieee.org](mailto:kongaik.lee@ieee.org)
+
+
+
+
+***Zhijie Yan, Principal Engineer at Alibaba, China***
+Email: [zhijie.yzj@alibaba-inc.com](mailto:zhijie.yzj@alibaba-inc.com)
+
+
+
+***Shiliang Zhang, Senior Engineer at Alibaba, China***
+Email: [sly.zsl@alibaba-inc.com](mailto:sly.zsl@alibaba-inc.com)
+
+
+
+***Yanmin Qian, Professor, Shanghai Jiao Tong University, China***
+
+Email: [yanminqian@sjtu.edu.cn](mailto:yanminqian@sjtu.edu.cn)
+
+
+
+***Zhuo Chen, Applied Scientist in Microsoft, USA***
+
+Email: [zhuc@microsoft.com](mailto:zhuc@microsoft.com)
+
+
+
+***Jian Wu, Applied Scientist in Microsoft, USA***
+
+Email: [wujian@microsoft.com](mailto:wujian@microsoft.com)
+
+
+
+***Hui Bu, CEO, AISHELL foundation, China***
+
+Email: [buhui@aishelldata.com](mailto:buhui@aishelldata.com)
+
+
diff --git a/docs/m2met2/Rules.md b/docs/m2met2/Rules.md
new file mode 100644
index 0000000..340c2f5
--- /dev/null
+++ b/docs/m2met2/Rules.md
@@ -0,0 +1,14 @@
+# Rules
+All participants should adhere to the following rules to be eligible for the challenge.
+
+- Data augmentation is allowed on the original training dataset, including, but not limited to, adding noise or reverberation, speed perturbation and tone change.
+
+- Participants are permitted to use the Eval set for model training, but it is not allowed to use the Test set for this purpose. Instead, the Test set should only be utilized for parameter tuning and model selection. Any use of the Test-2023 dataset that violates these rules is strictly prohibited, including but not limited to the use of the Test set for fine-tuning or training the model.
+
+- If the cpCER of the two systems on the Test dataset are the same, the system with lower computation complexity will be judged as the superior one.
+
+- If the forced alignment is used to obtain the frame-level classification label, the forced alignment model must be trained on the basis of the data allowed by the corresponding sub-track.
+
+- Shallow fusion is allowed to the end-to-end approaches, e.g., LAS, RNNT and Transformer, but the training data of the shallow fusion language model can only come from the transcripts of the allowed training dataset.
+
+- The right of final interpretation belongs to the organizer. In case of special circumstances, the organizer will coordinate the interpretation.
\ No newline at end of file
diff --git a/docs/m2met2/Track_setting_and_evaluation.md b/docs/m2met2/Track_setting_and_evaluation.md
new file mode 100644
index 0000000..2b75fca
--- /dev/null
+++ b/docs/m2met2/Track_setting_and_evaluation.md
@@ -0,0 +1,17 @@
+# Track & Evaluation
+## Speaker-Attributed ASR
+The speaker-attributed ASR task poses a unique challenge of transcribing speech from multiple speakers and assigning a speaker label to the transcription. Figure 2 illustrates the difference between the speaker-attributed ASR task and the multi-speaker ASR task. This track allows for the use of the AliMeeting, Aishell4, and Cn-Celeb datasets as constrained data sources during both training and evaluation. The AliMeeting dataset, which was used in the M2MeT challenge, includes Train, Eval, and Test sets. Additionally, a new Test-2023 set, consisting of approximately 10 hours of meeting data recorded in an identical acoustic setting as the AliMeeting corpus, will be released soon for challenge scoring and ranking. It's worth noting that the organizers will not provide the near-field audio, transcriptions, or oracle timestamps of the Test-2023 set. Instead, segments containing multiple speakers will be provided, which can be obtained using a simple voice activity detection (VAD) model.
+
+
+
+## Evaluation metric
+The accuracy of a speaker-attributed ASR system is evaluated using the concatenated minimum permutation character error rate (cpCER) metric. The calculation of cpCER involves three steps. Firstly, the reference and hypothesis transcriptions from each speaker in a session are concatenated in chronological order. Secondly, the character error rate (CER) is calculated between the concatenated reference and hypothesis transcriptions, and this process is repeated for all possible speaker permutations. Finally, the permutation with the lowest CER is selected as the cpCER for that session. TThe CER is obtained by dividing the total number of insertions (Ins), substitutions (Sub), and deletions(Del) of characters required to transform the ASR output into the reference transcript by the total number of characters in the reference transcript. Specifically, CER is calculated by:
+
+$$\text{CER} = \frac {\mathcal N_{\text{Ins}} + \mathcal N_{\text{Sub}} + \mathcal N_{\text{Del}} }{\mathcal N_{\text{Total}}} \times 100\%,$$
+
+where $\mathcal N_{\text{Ins}}$, $\mathcal N_{\text{Sub}}$, $\mathcal N_{\text{Del}}$ are the character number of the three errors, and $\mathcal N_{\text{Total}}$ is the total number of characters.
+## Sub-track arrangement
+### Sub-track I (Fixed Training Condition):
+Participants are required to use only the fixed-constrained data (i.e., AliMeeting, Aishell-4, and CN-Celeb) for system development. The usage of any additional data is strictly prohibited. However, participants can use open-source pre-trained models from third-party sources, such as [Hugging Face](https://huggingface.co/models) and [ModelScope](https://www.modelscope.cn/models), provided that the list of utilized models is clearly stated in the final system description paper.
+### Sub-track II (Open Training Condition):
+Participants are allowed to use any publicly available data set, privately recorded data, and manual simulation to build their systems in addition to the fixed-constrained data. They can also utilize any open-source pre-trained models, but it is mandatory to provide a clear list of the data and models used in the final system description paper. If manually simulated data is used, a detailed description of the data simulation scheme must be provided.
\ No newline at end of file
diff --git a/docs/m2met2/conf.py b/docs/m2met2/conf.py
new file mode 100644
index 0000000..6edc1b4
--- /dev/null
+++ b/docs/m2met2/conf.py
@@ -0,0 +1,44 @@
+# Configuration file for the Sphinx documentation builder.
+#
+# For the full list of built-in configuration values, see the documentation:
+# https://www.sphinx-doc.org/en/master/usage/configuration.html
+
+# -- Project information -----------------------------------------------------
+# https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information
+
+project = "MULTI-PARTY MEETING TRANSCRIPTION CHALLENGE 2.0"
+copyright = "2023, Speech Lab, Alibaba Group; ASLP Group, Northwestern Polytechnical University"
+author = "Speech Lab, Alibaba Group; Audio, Speech and Language Processing Group, Northwestern Polytechnical University"
+
+
+extensions = [
+ "nbsphinx",
+ "sphinx.ext.autodoc",
+ "sphinx.ext.napoleon",
+ "sphinx.ext.viewcode",
+ "sphinx.ext.mathjax",
+ "sphinx.ext.todo",
+ # "sphinxarg.ext",
+ "sphinx_markdown_tables",
+ # 'recommonmark',
+ "sphinx_rtd_theme",
+ "myst_parser",
+]
+
+myst_enable_extensions = [
+ "colon_fence",
+ "deflist",
+ "dollarmath",
+ "html_image",
+]
+
+myst_heading_anchors = 2
+myst_highlight_code_blocks = True
+myst_update_mathjax = False
+
+templates_path = ["_templates"]
+source_suffix = [".rst", ".md"]
+
+pygments_style = "sphinx"
+
+html_theme = "sphinx_rtd_theme"
diff --git a/docs/m2met2/images/buhui.jpeg b/docs/m2met2/images/buhui.jpeg
new file mode 100644
index 0000000..86efc8f
Binary files /dev/null and b/docs/m2met2/images/buhui.jpeg differ
diff --git a/docs/m2met2/images/chenzhuo.jpg b/docs/m2met2/images/chenzhuo.jpg
new file mode 100644
index 0000000..66aaad2
Binary files /dev/null and b/docs/m2met2/images/chenzhuo.jpg differ
diff --git a/docs/m2met2/images/dataset_details.png b/docs/m2met2/images/dataset_details.png
new file mode 100644
index 0000000..c9ac1c0
Binary files /dev/null and b/docs/m2met2/images/dataset_details.png differ
diff --git a/docs/m2met2/images/kong.png b/docs/m2met2/images/kong.png
new file mode 100644
index 0000000..c81b172
Binary files /dev/null and b/docs/m2met2/images/kong.png differ
diff --git a/docs/m2met2/images/lxie.jpeg b/docs/m2met2/images/lxie.jpeg
new file mode 100644
index 0000000..477cfda
Binary files /dev/null and b/docs/m2met2/images/lxie.jpeg differ
diff --git a/docs/m2met2/images/meeting_room.png b/docs/m2met2/images/meeting_room.png
new file mode 100644
index 0000000..d26178d
Binary files /dev/null and b/docs/m2met2/images/meeting_room.png differ
diff --git a/docs/m2met2/images/qian.jpeg b/docs/m2met2/images/qian.jpeg
new file mode 100644
index 0000000..61d4af3
Binary files /dev/null and b/docs/m2met2/images/qian.jpeg differ
diff --git a/docs/m2met2/images/sa_asr_arch.png b/docs/m2met2/images/sa_asr_arch.png
new file mode 100644
index 0000000..fd65dd2
Binary files /dev/null and b/docs/m2met2/images/sa_asr_arch.png differ
diff --git a/docs/m2met2/images/task_diff.png b/docs/m2met2/images/task_diff.png
new file mode 100644
index 0000000..5cb6c0a
Binary files /dev/null and b/docs/m2met2/images/task_diff.png differ
diff --git a/docs/m2met2/images/wujian.jpg b/docs/m2met2/images/wujian.jpg
new file mode 100644
index 0000000..4d968d3
Binary files /dev/null and b/docs/m2met2/images/wujian.jpg differ
diff --git a/docs/m2met2/images/zhijie.jpg b/docs/m2met2/images/zhijie.jpg
new file mode 100644
index 0000000..11cfabf
Binary files /dev/null and b/docs/m2met2/images/zhijie.jpg differ
diff --git a/docs/m2met2/images/zsl.JPG b/docs/m2met2/images/zsl.JPG
new file mode 100644
index 0000000..c6318ac
Binary files /dev/null and b/docs/m2met2/images/zsl.JPG differ
diff --git a/docs/m2met2/index.rst b/docs/m2met2/index.rst
new file mode 100644
index 0000000..6567dee
--- /dev/null
+++ b/docs/m2met2/index.rst
@@ -0,0 +1,26 @@
+.. m2met2 documentation master file, created by
+ sphinx-quickstart on Tue Apr 11 14:18:55 2023.
+ You can adapt this file completely to your liking, but it should at least
+ contain the root `toctree` directive.
+
+ASRU 2023 MULTI-CHANNEL MULTI-PARTY MEETING TRANSCRIPTION CHALLENGE 2.0 (M2MeT2.0)
+==================================================================================
+Building on the success of the M2MeT challenge, we are delighted to propose the M2MeT2.0 challenge as a special session at ASRU2023.
+To advance the current state-of-the-art in multi-talker automatic speech recognition, the M2MeT2.0 challenge proposes a speaker-attributed ASR task, comprising two sub-tracks: fixed and open training conditions.
+To facilitate reproducible research, we provide a comprehensive overview of the dataset, challenge rules, evaluation metrics, and baseline systems.
+
+Now the new test set contains about 10 hours audio is available. You can download from `here