Preprint / Version 1

EWS: the Economic Watcher Survey Datasets and Tasks for the Financial and Economic Domain

##article.authors##

DOI:

https://doi.org/10.51094/jxiv.842

Keywords:

Dataset, Japanese, Sentence Classification, Financial Text Mining

Abstract

We construct a large dataset corresponding to three financial and economic domain text classification tasks, including sentiment analysis, using the Economy Watchers Survey.The Economy Watchers Survey is a crucial data source released monthly by the Cabinet Office to swiftly grasp the economic situation in Japan.We ensure that the latest task datasets are always available by building a framework to automatically integrate and release the monthly survey results.

Conflicts of Interest Disclosure

The authors declare no conflict of interest.

Downloads *Displays the aggregated results up to the previous day.

Download data is not yet available.

References

T. Jayakumar, F. Farooqui, and L. Farooqui, "Large Language Models are legal but they are not: Making the case for a powerful LegalLLM," Proceedings of the Natural Legal Language Processing Workshop 2023, pp.223–229, Association for Computational Linguistics, Singapore, Dec. 2023. https://aclanthology.org/2023.nllp-1.22

K. Singhal, S. Azizi, T. Tu, S.S. Mahdavi, J. Wei, H.W. Chung, N. Scales, A. Tanwani, H. Cole-Lewis, S. Pfohl, et al., "Large language models encode clinical knowledge," Nature, vol.620, no.7972, pp.172–180, 2023.

T. Brown, B. Mann, N. Ryder, M. Subbiah, J.D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al., "Language models are few-shot learners," Advances in neural information processing systems, vol.33, pp.1877–1901, 2020.

T. Kojima, S.S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa, "Largelanguage models are zero-shot reasoners," Advances in neural information processing systems, vol.35, pp.22199–22213, 2022.

X. Li, S. Chan, X. Zhu, Y. Pei, Z. Ma, X. Liu, and S. Shah, "Are ChatGPT and GPT-4 General-Purpose Solvers for Financial Text Analytics? A Study on Several Typical Tasks," Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Industry Track, Singapore, Dec. 2023. https://aclanthology.org/2023.emnlp-industry.39

L. Gao, J. Tow, B. Abbasi, S. Biderman, S. Black, A. DiPofi, C. Foster, L. Golding, J. Hsu, A. Le Noac’h, H. Li, K. McDonell, N. Muennighoff, C. Ociepa, J. Phang, L. Reynolds, H. Schoelkopf, A. Skowron, L. Sutawika, E. Tang, A. Thite, B. Wang, K. Wang, and A. Zou, "A framework for few-shot language model evaluation," Dec. 2023. https://zenodo.org/records/10256836

N. Guha, J. Nyarko, D.E. Ho, C. R´e, A. Chilton, A. Narayana, A. Chohlas-Wood, A. Peters, B. Waldon, D.N. Rockmore, D. Zambrano, D. Talisman, E. Hoque, F. Surani, F. Fagan, G. Sarfaty, G.M. Dickinson, H. Porat, J. Hegland, J. Wu, J. Nudell, J. Niklaus, J. Nay, J.H. Choi, K. Tobia, M. Hagan, M. Ma, M. Livermore, N. Rasumov-Rahe, N. Holzenberger, N. Kolt, P. Henderson, S. Rehaag, S. Goel, S. Gao, S. Williams, S. Gandhi, T. Zur, V. Iyer, and Z. Li, "LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models," 2023. https://arxiv.org/abs/2308.11462

Q. Xie, W. Han, Z. Chen, R. Xiang, X. Zhang, Y. He, M. Xiao, D. Li, Y. Dai, D. Feng, Y. Xu, H. Kang, Z. Kuang, C. Yuan, K. Yang, Z. Luo, T. Zhang, Z. Liu, G. Xiong, Z. Deng, Y. Jiang, Z. Yao, H. Li, Y. Yu, G. Hu, J. Huang, X.-Y. Liu, A. Lopez-Lira, B. Wang, Y. Lai, H. Wang, M. Peng, S. Ananiadou, and J. Huang, "FinBen: A Holistic Financial Benchmark for Large Language Models," 2024. https://arxiv.org/abs/2402.12659

M. Hirano, "Construction of a Japanese Financial Benchmark for Large Language Models," Proceedings of the Joint Workshop of the 7th Financial Technology and Natural Language Processing, the 5th Knowledge Discovery from Unstructured Data in Financial Services, and the 4th Workshop on Economics and Natural Language Processing @ LREC-COLING 2024, pp.1–9, ELRA and ICCL, Torino, Italia, May 2024. https://aclanthology.org/2024.finnlp-1.1

Y. Dai, D. Feng, J. Huang, H. Jia, Q. Xie, Y. Zhang, W. Han, W. Tian, and H. Wang, "LAiW: A Chinese Legal Large Language Models Benchmark," 2024. https://arxiv.org/abs/2310.05620

K. Goshima, H. Ishijima, M. Shintani, and H. Yamamoto, "Forecasting Japanese inflation with a news-based leading indicator of economic activities," Studies in Nonlinear Dynamics & Econometrics, vol.25, no.4, pp.111–133, 2021.

J. Nakajima, H. Yamagata, T. Okuda, S. Katsuki, and T. Shinohara, "Extracting firms’ short-term inflation expectations from the economy watchers survey using text analysis," Technical report, Bank of Japan, 2021.

K. Seki, Y. Ikuta, and Y. Matsubayashi, "News-based business sentiment and its properties as an economic index," Information Processing & Management, vol.59, no.2, p.102795, 2022. https://www.sciencedirect.com/science/article/pii/S0306457321002739

R. Shah, K. Chawla, D. Eidnani, A. Shah, W. Du, S. Chava, N. Raman, C. Smiley, J. Chen, and D. Yang, "When FLUE meets FLANG: Benchmarks and large pretrained language model for financial domain," Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, eds. by Y. Goldberg, Z. Kozareva, and Y. Zhang, pp.2322–2335, Association for Computational Linguistics, Abu Dhabi, United Arab Emirates, Dec. 2022. https://aclanthology.org/2022.emnlp-main.148

P. Malo, A. Sinha, P. Korhonen, J. Wallenius, and P. Takala, "Good debt or bad debt: Detecting semantic orientations in economic texts," Journal of the Association for Information Science and Technology, vol.65, no.4, pp.782–796, 2014.

M. Maia, S. Handschuh, A. Freitas, B. Davis, R. McDermott, M. Zarrouk, and A. Balahur, "WWW’18 Open Challenge: Financial Opinion Mining and Question Answering," Companion Proceedings of the The Web Conference 2018, p.1941–1942, 2018. https://doi.org/10.1145/3184558.3192301

A. Sinha and T. Khandait, "Impact of news on the commodity market: Dataset and results," Advances in Information and Communication: Proceedings of the 2021 Future of Information and Communication Conference (FICC), Volume 2, pp.589–601, 2021. https://doi.org/10.1007/978-3-030-73103-8 41

J.C.S. Alvarado, K. Verspoor, and T. Baldwin, "Domain adaption of named entity recognition to support credit risk assessment," Proceedings of the Australasian Language Technology Association Workshop 2015, pp.84–90, 2015. https://aclanthology.org/U15-1010

A. Shah, S. Paturi, and S. Chava, "Trillion Dollar Words: A New Financial Dataset, Task & Market Analysis," Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp.6664–6679, Association for Computational Linguistics, Toronto, Canada, July 2023. https://aclanthology.org/2023.acl-long.368

M. Suzuki, H. Sakaji, M. Hirano, and K. Izumi, "Constructing and analyzing domain-specific language model for financial text mining," Information Processing & Management, vol.60, no.2, p.103194, 2023.

鈴木 雅弘，坂地 泰紀，平野 正徳，和泉 潔，"FinDeBERTaV2: 単語分割フリーな金融事前学習言語モデル," 人工知能学会論文誌，vol.39，no.4，pp.FIN23–G 1–14，2024．

増田 樹，中川 慧，星野 崇宏，"ChatGPT は公認会計士試験を突破できるか？: 短答式試験監査論への挑戦," 第 31回 人工知能学会 金融情報学研究会（SIG-FIN），pp.81–88，2023．

D. Bragoli, "Now-casting the japanese economy," International Journal of Forecasting, vol.33, no.2, pp.390–402, 2017. https://www.sciencedirect.com/science/article/pii/S0169207016301297

Downloads

Posted


Submitted: 2024-08-06 06:33:42 UTC

Published: 2024-08-08 09:02:04 UTC
Section
Information Sciences

License

Copyright (c) 2024 Masahiro Suzuki
Hiroki Sakaji

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.