Projects

Multi-Agent LLM System for Automated Manufacturing Decision-Making and Cost Estimation

1 / 2

Problem Statement: Traditional manufacturing assessment and quotation generation processes are time-consuming, costly, and lack consistency

2 / 2

Proposed Solution: Agentic RAG system powered by specialized LLMs for automated manufacturing assessment and decision-making

❮ ❯

This ongoing research addresses a critical challenge in manufacturing: the labor-intensive, error-prone, and time-consuming nature of manual decision-making processes for machine tool selection, cost estimation, and quotation generation. We propose a novel multi-agent framework leveraging Large Language Models (LLMs) and Vision-Language Models (VLMs) to automate these processes.

Our approach integrates specialized agents for engineering drawing interpretation, CAD model feature recognition, and manufacturing knowledge retrieval through Retrieval-Augmented Generation (RAG). The system grounds its recommendations in factual knowledge bases while providing consistent, efficient manufacturing decision support with detailed cost estimations, quotation generation, and process planning.

This research is being conducted with the support of the Sectoral AI Centre of Excellence for Manufacturing (AIMfg) in A*STAR, Singapore, and aims to significantly reduce the time and expertise required for manufacturing assessment while improving consistency and accuracy.

Fine-Tuning Vision-Language Model for Automated Engineering Drawing Information Extraction

Khan, M.T., Chen, L., Ng, Y., Feng, W., Tan, N.Y.J., and Moon, S.K., 2024. Fine-Tuning Vision-Language Model for Automated Engineering Drawing Information Extraction. 9th International Conference on Innovation in Artificial Intelligence (ICIAI 2025).

1 / 2

Fine-tuned vision-language model for engineering drawing information extraction

2 / 2

GD&T Recognition Data Example

❮ ❯

Leveraging Vision-Language Models for Manufacturing Feature Recognition in CAD Designs

Dataset and Code:

• MFCAD-VLM Dataset: Manufacturing CAD Feature Dataset with STEP Files, Ground Truth JSON Annotations, and Multi-View Isometric Images for Automatic Feature Recognition
• VLM-CADFeatureRecognition Code: code and resources for automating manufacturing feature recognition in CAD designs using vision-language models.

Publication:

Khan, M.T., Chen, L., Ng, Y.H., Feng, W., Tan, N.W.J., and Moon, S.K., 2024. Leveraging Vision-Language Models for Manufacturing Feature Recognition in CAD Designs. ASME Journal of Computing and Information Science in Engineering (JCISE), Under Review.

1 / 4

Overview of Vision-Language Model approach for manufacturing feature recognition in CAD designs

2 / 4

Few-shot prompt engineering strategy for instructing VLMs to identify manufacturing features

3 / 4

MFCAD-VLM dataset: A comprehensive collection of CAD models with annotated manufacturing features

4 / 4

Evaluation results and performance comparison of various Vision-Language Models on challenging CAD examples

❮ ❯

Automatic Feature Recognition and Dimensional Attributes Extraction from CAD Models for Hybrid Additive-Subtractive Manufacturing

Khan, M.T., Feng, W., Chen, L., Ng, Y., Tan, N.Y.J., and Moon, S.K., 2024. Automatic Feature Recognition and Dimensional Attributes Extraction from CAD Models for Hybrid Additive-Subtractive Manufacturing. ASME 2024 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference (IDETC CIE2024), Boston, USA.

1 / 10

HybridCAD++ Examples

2 / 10

3D CAD feature recognition system highlighting detected manufacturing features

3 / 10

Automatic Feature Recognition (AFR) results showing detected features with dimensional attributes

4 / 10

HybridCAD++ Feature List

5 / 10

HybridCAD++ Feature List

6 / 10

HybridCAD++ Feature List

7 / 10

HybridCAD++ Feature List

8 / 10

HybridCAD++ Feature List

9 / 10

HybridCAD++ Feature List

10 / 10

HybridCAD++ Feature List

❮ ❯

Audio-Visual Cross-Modality Knowledge Transfer for Machine Learning-Based In-Situ Monitoring in Laser Additive Manufacturing

Xie, J., Safdar, M., Chen, L., Moon, S.K., and Zhao, Y.F., 2024. Audio-Visual Cross-Modality Knowledge Transfer for Machine Learning-Based In-Situ Monitoring in Laser Additive Manufacturing. Additive Manufacturing, Under Review.

Audio-Visual Monitoring Setup for Laser Additive Manufacturing

Cross-Modality Knowledge Transfer Framework

Synchronized and registered audio data (converted to spectrograms) and visual data (melt pool images)

Schematics of the proposed cross-modal knowledge transfer (CMKT) based on contrastive and classification semantic alignment (CCSA)

Examples of positive masks (Plime) in yellow generated using LIME for (a) the visual modality (melt pool images) and (b) the audio modality (spectrograms)

❮ ❯

Multi-scale Defect Detection and Adaptive Mitigation for Defect-Free Autonomous Laser-Directed Energy Deposition

1 / 3

Integrated end-to-end workflow for multi-scale defect detection, predictive forecasting, and defect mitigation in laser-directed energy deposition

2 / 3

Hierarchical multi-scale, multi-sensor monitoring architecture for WL-DED process

3 / 3

Proactive next-layer defect and anomaly forecasting approach using spatiotemporal learning

❮ ❯

This research focuses on developing advanced monitoring and control capabilities for wire laser-directed energy deposition (WL-DED) processes. The project aims to address critical challenges in additive manufacturing by integrating hierarchical multi-sensor monitoring with predictive analytics and adaptive process control.

By transitioning from reactive defect detection to proactive quality forecasting, this work enables dynamic in-process adjustments to prevent defect formation before it occurs. The approach combines real-time acoustic, thermal, and vision monitoring with spatiotemporal deep learning models to anticipate potential anomalies in subsequent layers.

This paradigm shift in online quality assurance for L-DED helps minimize material waste, reduce operational downtime, and significantly improve product quality, ultimately paving the way toward sustainable, defect-free, autonomous additive manufacturing.

In-Situ Process Monitoring and Adaptive Quality Enhancement in Laser Additive Manufacturing: A Critical Review

GitHub Repository

Awesome-AM-process-monitoring-control: A curated collection of research papers with open-source implementations/datasets focused on in-situ process monitoring and adaptive control in laser-based additive manufacturing.

Publication

Chen, L., Bi, G., Yao, X., Su, J., Tan, C., Feng, W., Chew, Y. and Moon, S.K., 2024. In-situ process monitoring and adaptive quality enhancement in laser additive manufacturing: a critical review. Journal of Manufacturing Systems, vol. 74, Pages 527-574.

1 / 4

Overview of in-situ process monitoring and adaptive quality enhancement in Laser AM

2 / 4

Coaxial melt pool visualization techniques showing feature extraction, thermal characteristics, and size variations across different build geometries

3 / 4

Acoustic signal signatures and their correlation to various defect types in laser powder bed fusion processes

4 / 4

In-process adaptive dimension correction using laser scanning, machine learning-based defect identification, and repair strategy implementation

❮ ❯

Laser Wire-Directed Energy Deposition (LW-DED) of Al7075: Meltpool Monitoring and Anomaly Detection

1 / 4

Schematic of coaxial wire-fed LW-DED system with integrated audio-thermal monitoring

2 / 4

Coaxial Wire-Laser Directed Energy Deposition system for Al7075 processing

3 / 4

Process parameter map showing defect formation regimes in laser-wire deposition

4 / 4

Characteristic audio-visual sensor signatures during LW-DED defect formation

❮ ❯

Multisensor Fusion-Based Digital Twin for Localized Quality Prediction in Robotic Laser-Directed Energy Deposition

Chen, L., Bi, G., Yao, X., Tan, C., Su, J., Ng, N.P.H., Chew, Y., Liu, K. and Moon, S.K., 2023. Multisensor fusion-based digital twin for localized quality prediction in robotic laser-directed energy deposition. Robotics and Computer-Integrated Manufacturing, 84, p.102581.

LDEDFusionNet Dataset

Related Repositories

• ROS-Based Multi-Sensor Monitoring Software Platform for Laser AM: Real-time deployment of trained ML models in a ROS environment.
• LDED-FusionNet: Machine Learning-Based Audio-Visual Defect Detection for LDED AM Process: Implementation of audio-visual feature fusion using machine learning for defect detection and quality prediction in Laser-Directed Energy Deposition (LDED) additive manufacturing processes.
• LDED-Multisensor-Fusion: Multisensor Fusion-Based Digital Twin for Laser-Directed Energy Deposition (LDED): Original implementation for multisensor fusion-based digital twin modeling designed specifically for the Laser-Directed Energy Deposition (LDED) additive manufacturing (AM) process. The methodology integrates audio-visual sensor data fusion and machine learning (ML) techniques to enable accurate localized quality prediction and defect detection. Additionally, it includes comprehensive tools for constructing a virtual quality map that aligns multisensor data with robotic spatial data.
• MultiSensor-Monitoring-LW-DED: Thermal-acoustic monitoring code tailored for laser-wire DED processes.

Related Publications

1. Multisensor Fusion-Based Digital Twin for Localized Quality Prediction in Robotic Laser-Directed Energy Deposition (RCIM, 2023) Link
   • Proposes a Multisensor Fusion-Based Digital Twin, leveraging feature-level fusion of acoustic and visual data for LDED quality prediction.
   • Demonstrates significant improvements in localized quality prediction.
2. In-situ Defect Detection in Laser-Directed Energy Deposition with Machine Learning and Multi-Sensor Fusion (JMST, 2024) Link
   • Explores acoustic signals and coaxial melt pool images for defect detection.
   • Presents intra-modality and cross-modality feature correlations to identify critical audiovisual signatures in LDED process dynamics.
3. Inference of Melt Pool Visual Characteristics in Laser Additive Manufacturing Using Acoustic Signal Features and Robotic Motion Data (ICCAR, 2024) Link
   • Proposes a novel technique to infer melt pool visual characteristics in LAM by combining acoustic signal features with robotic tool-center-point (TCP) motion data.
   • Highlights the potential of microphone-based monitoring as a cost-effective alternative for melt pool tracking and closed-loop control in LAM
4. Multimodal Sensor Fusion for Real-Time Location-Dependent Defect Detection in Laser-Directed Energy Deposition (IDETC-CIE, 2023) Link
   • Utilizes a hybrid CNN to directly fuse acoustic and visual raw data.
   • Achieves high defect detection accuracy without manual feature extraction.

1 / 5

Multi-sensor fusion-based digital twin (MFDT) framework for robotic laser-directed energy deposition

2 / 5

Physical implementation of the multi-sensor monitoring setup for laser-directed energy deposition

3 / 5

Digital twin visualization showing spatiotemporal multi-sensor features

4 / 5

Graphical user interface for real-time multimodal monitoring and data fusion

5 / 5

Virtual quality prediction map showing localized defect probability across the build

❮ ❯

Acoustic-Based Defect Detection in Laser-DED Through Deep Learning

Chen, L., Yao, X., Tan, C., He, W., Su, J., Weng, F., Chew, Y., Ng, N.P.H. and Moon, S.K., 2023. In-situ crack and keyhole pore detection in laser directed energy deposition through acoustic signal and deep learning. Additive Manufacturing, 103547.

1 / 4

In-situ defect detection in LDED through acoustic signal and deep learning

2 / 4

Comprehensive dataset of acoustic signals captured during LDED process with labeled defect classifications

3 / 4

Time-frequency analysis of acoustic signatures associated with different defect types in LDED

4 / 4

Confusion matrix showing classification performance of the deep learning model for defect detection

❮ ❯

In-situ Surface Quality Monitoring in Laser-DED

Chen, L., Yao, X., Xu, P., Moon, S.K. and Bi, G., 2020. Rapid surface defect identification for additive manufacturing with in-situ point cloud processing and machine learning. Virtual and Physical Prototyping, 16(1), pp.50-67.

AI-assisted surface defect detection

Adaptive Dimension Correction Strategy for Laser-Aided Additive Manufacturing

Xu, P., Yao, X., Chen, L., Zhao, C., Liu, K., Moon, S.K. and Bi, G., 2022. In-process adaptive dimension correction strategy for laser aided additive manufacturing using laser line scanning. Journal of Materials Processing Technology, 303, p.117544.

In-process adaptive dimension correction strategy

Data-Driven Adaptive Control System for Laser-DED

Chen, L., Yao, X., Chew, Y., Weng, F., Moon, S.K. and Bi, G., 2020. Data-driven adaptive control for laser-based additive manufacturing with automatic controller tuning. Applied Sciences, 10(22), Art. no. 22.

PropelPath: High-Speed Directed Energy Deposition Toolpath Strategy for Propeller Manufacturing

Chen, L., Ng, Y.H., Liu, S., Sun, X., Choy, C.K., Chen, P., and Chew, Y., 2023. PropelPath: A High-Speed Directed Energy Deposition Toolpath Strategy for Propeller Manufacturing with Complex Geometries. Technology Disclosure, Advanced Remanufacturing and Technology Centre.

PropelPath Algorithm Workflow

Advanced Toolpath Solutions

Process Optimization Results

❮ ❯

PropelPath is a novel toolpath generation strategy specifically designed for laser-wire directed energy deposition (LW-DED) and wire-arc additive manufacturing (WAAM) processes. The technology addresses the unique challenges of fabricating complex geometries like marine propellers, where conventional “zig-zag” toolpath strategies fall short.

Key innovations include:

• Edge-parallel continuous curved toolpath strategy that dynamically adapts to complex part geometries
• Selective material placement at critical regions (sharp edges, high curvature areas) to prevent defects
• Adaptive infill algorithm that adjusts to varying geometrical characteristics across layers
• Comprehensive solution for high aspect-ratio structures and overhang regions
• Significant reduction in material waste through targeted deposition compared to uniform expansion methods

The toolpath generation algorithm includes multiple steps of optimization: boundary extraction, strategic infill line definition, edge consistency adjustments, and dynamic parameter adaptation. A graphical user interface (GUI) was developed to simplify configuration of slicing, infill strategy, and edge adjustments.

This technology has been implemented in prototype form and experimentally validated, demonstrating enhanced process stability, improved fabrication quality, and reduced material usage (more than 50%) compared to conventional approaches.

AI-Assisted Material Design for Advanced Manufacturing

Tan, C., Li, Q., Yao, X., Chen, L., Su, J., Ng, F.L., Liu, Y., Yang, T., Chew, Y., Liu, C.T. and DebRoy, T., 2023. Machine Learning Customized Novel Material for Energy-Efficient 4D Printing. Advanced Science, 10(1), 2206607.

Material Design Framework

↑

Additive Manufacturing Metallurgy Guided Machine Learning Design of Versatile Alloys

Jinlong Su, Lequn Chen, Steven Van Petegem, Fulin Jiang, Qinzhi Li, Junhua Luan, Swee Leong Sing, Jian Wang, Chaolin Tan (Submitted, under review)

1 / 5

Framework for AM metallurgy-guided machine learning design of martensitic steel

2 / 5

ML-based surrogate modeling with multi-objective optimization for alloy design

3 / 5

Parallel coordinates visualization of Pareto front solutions

4 / 5

Two-dimensional visualization of Pareto solutions with TOPSIS-selected optimal composition

5 / 5

Comparison between optimized alloy and other commercial high-strength steels

❮ ❯

This research presents a novel approach to designing advanced alloys for additive manufacturing by integrating metallurgical principles with machine learning techniques. We developed a comprehensive framework that builds surrogate models to predict key material properties and processability factors from alloy compositions.

Our methodology employs multi-objective optimization to navigate the complex trade-offs between various targets, generating a Pareto front of optimal compositions. Using the TOPSIS decision-making algorithm, we identified an optimal composition that outperforms commercial alternatives.