Japan Digital Design Develops LLM Specialized in Financial Market Translation
Japan Digital Design (JDD) has developed a 70-billion parameter large language model (LLM) called "Llama-3.3-fintranslate-JDD-70b" specialized in translating financial market-related documents. In internal evaluations using actual business data, the model achieved performance exceeding global models such as GPT-4o. Additionally, JDD plans to soon begin providing a translation application utilizing this model to MUFG Bank.
Background: Financial Market-Specific Documents Are Difficult to Translate with Existing Tools
In financial market-related operations that are globally interconnected, gathering information from around the world and disseminating Japanese information globally is a crucial part of the work. Efficiently translating various daily financial market-related documents such as market reports, news, and central bank releases between English and Japanese is important for improving employee operational efficiency and strengthening information dissemination capabilities. However, existing translation tools have struggled to properly translate expressions specific to financial market-related documents.
JDD's Solution: Building a Proprietary LLM Trained on MUFG Bank's Market Reports
To address these challenges, JDD constructed a parallel dataset from Japanese and English financial market reports previously created by MUFG Bank. Using "Llama-3.3-Swallow-70B-v0.4"—which improved the Japanese performance of Meta Platforms' "Llama-3.3-70B-Instruct"—as the base model, JDD implemented continued pre-training and supervised fine-tuning to build a model specialized in translating financial market-related documents.
The aim is to reduce the workload of staff by generating business-ready translations with this proprietary model.
Performance Verification #1
Performance evaluation of JDD's market translation-specialized model was conducted by experts engaged in MUFG Bank's market-related operations. Fifteen paragraphs were extracted from multiple financial market reports that are actual translation targets in business operations. The breakdown included 10 English-to-Japanese translations and 5 Japanese-to-English translations, with both relative and absolute evaluations requested from 4 experts per question, yielding a total of 48 valid responses.
In relative evaluation, approximately 60% judged JDD's model translations as superior to GPT-4o. In absolute evaluation, 93% determined that JDD's model translations could be used in business operations with minor or no human modifications.
Performance Verification #2
Translation texts were created for approximately 4,500 financial market-related documents using both JDD's model and GPT-4o. GPT-4.1 was used to score each generated translation on the following 5 items on a 4-point scale:
- FLUENCY: How natural it is in the target language
- ADEQUACY: How correctly the original meaning is conveyed
- MARKET_STYLE: Correct selection of specialized terminology used in financial markets
- COVERAGE: Translation without omissions from the original content
- OVERALL: Comprehensive translation quality based on the above 4 items
JDD's model surpassed GPT-4o in items such as FLUENCY, STYLE, and ADEQUACY, and was confirmed to exceed GPT-4o in OVERALL as well.
Implementation Process and Outlook
This project, developed over a 6-month period in collaboration with MUFG Bank's Treasury and Securities Division and Digital Strategy Division, began as R&D to accumulate LLM utilization know-how for both MUFG Bank and JDD. Thanks to assistance from all parties involved, the performance of the developed translation-specialized LLM has reached a level suitable for business application, and JDD plans to provide it to MUFG Bank as an application.
Leveraging the business insights and technical assets gained from this initiative, JDD aims to further enhance business support solutions utilizing generative AI technology while continuing to contribute to accelerating business DX in the financial industry.
About Japan Digital Design (JDD)
Under the mission of "creating a new normal in finance and contributing to people's growth," JDD is a member of the Mitsubishi UFJ Financial Group (MUFG) and aims to create better financial experiences.
JDD provides solutions for MUFG's DX activities, including customer-centric experience design, advanced data analysis and AI/machine learning model development, and flexible system design and implementation incorporating the latest technologies. Through collaboration with MUFG, personnel exchanges, and R&D functions, JDD leads financial innovation and continuously updates the financial experience for customers and MUFG's business environment.
Norbert Gehrke
