Research Focus and Philosophy 

I have been known internationally for my work on the research and development of Adaptive Resonance Theory (ART) models and algorithms for machine learning, multimodal information mining, and high level cognitive modeling.

Adaptive Resonance Theory (ART) models, also known as self-organizing neural networks, are biologically-inspired computational models capable of performing fast and stable pattern recognition, memory, knowledge discovery and a myriad of autonomous processes.

My research work centered on Adaptive Resonance Theory is by nature interdisciplinary, spanning several interrelated fields, including neural networks, cognitive science, computational neuroscience, artificial intelligence, knowledge discovery, machine learning, text mining, information fusion, intelligent agents and personalization.

My key research contributions can be broadly organized into two core areas, namely (1) Cognitive and Neural Systems; and (2) Multimodal Information Mining and Fusion. This research statement shall review my key contributions in these two core research areas, along with three tracks of major projects I have undertaken: The first track is a series of four back-to-back projects with DSO National Laboratories in developing Biologically-Inspired Cognitive Architectures. The second track is a project Agent Augmented Co-Space, funded by National Research Foundation’s Interactive Digital Media (IDM) programme office through Media Development Authority (MDA). The third track is a project called Aging in Place, funded by NRF under Joint UBC-NTU Research Centre of Excellent on Active Living (LILY), for development of assistive technologies for supporting the active living of elderly.

My research impacts are in terms of publications, technology transfer, patents and consultancy. Being an academic, I fully understand the need to publish in top-notch journals and conferences. On the other hand, I firmly believe research is for the purpose of innovation and creation of new technologies. As such, publication alone is never the primary aim of my research projects. Constantly motivated by real world practical problems, I have been researching into new designs, methods and technologies that are usable in real-life environment. Consequently, my research work always results in demonstrable prototypes and licensable technologies and systems.

Research Contributions

Contributions to Foundation of Cognitive and Neural Systems

Besides the domains of text mining and information fusion,

my long-term research interest has been on developing biologically-inspired neural architectures with high level cognitive capabilities.

My doctoral thesis research at Boston University was on the integration of symbolic and neural systems. Specifically, I showed that the famous class of self-organizing neural networks, known as Adaptive Resonance Theory (ART), can be generalized naturally into a bidirectional model called Adaptive Resonance Associate Map [TAN95] to support both supervised learning and associative recall. In addition, the resultant predictive ART architecture is compatible with symbolic rule-based knowledge and can be used for domain knowledge insertion, refinement, and extraction [CT95; TAN97]. This research result has important implications as it lays the foundation for intelligent systems that integrate the complementary strengths of symbolic and neural systems. My PhD research was further expanded as part of a collaboration project with the Japan’s Real World Computing partnership (RWCP, http://www.rwcp.or.jp) from 1994 to 1997. The key papers published during the period contained many important results, which subsequently spurred my work in brain-inspired cognitive systems.

My more recent work on cognitive and neural system research has focused on integrated cognitive architectures, with capabilities of self-awareness, intention, learning, reasoning, and surprise handling. Following the embodied cognition approach, a further generalization of the ART model leads to a learning model called Fusion Architecture for Learning and Cognition (FALCON) that enables an agent to operate and learn in a dynamic environment through reinforcement learning [TAN04; TLX08; TTZ15; XT07; XT13; WT15]. More importantly, the generalized fusion ART model [TCG07] provides new insights into how various memory modules in the human brain interact with each other and forms the foundation of emergent high-level cognition.

Selected Publications

[CT95]  Gail A. Carpenter and Ah-Hwee Tan. Rule Extraction: From Neural Architecture to Symbolic Representation. Connection Science, Vol. 7, No. 1 (1995) 3-27.

[TAN95] Ah-Hwee Tan. Adaptive Resonance Associative Map. Neural Networks, Vol.8, No. 3 (1995) 437-446.

[TAN97] Ah-Hwee Tan. Cascade ARTMAP: Integrating Neural Computation and Symbolic Knowledge Processing. IEEE Transactions on Neural Networks, Vol. 8, No. 2 (March 1997) 237-250.

[TCG07] Ah-Hwee Tan, Gail A. Carpenter and Stephen Grossberg. Intelligence Through Interaction: Towards A Unified Theory for Learning. In proceedings, D. Liu et al. (Eds.): LNCS 4491, Part I, pp. 1098-1107, 2007.

[TP05]  Ah-Hwee Tan and Hong Pan. Predictive Neural Networks for Gene Expression Data Analysis. Neural Networks, Vol. 18, No. 3 (April 2005) 297-306.

[TLX08] Ah-Hwee Tan, Ning Lu and Dan Xiao. Integrating Temporal Difference Methods and Self-Organizing Neural Networks for Reinforcement Learning with Delayed Evaluative Feedback. IEEE Transactions on Neural Networks, Vol. 9, No. 2 (February 2008) 230-244.

[TTZ15] Teck-Hou Teng, Ah-Hwee Tan and Jacek Zurada. Self-Organizing Neural Networks Integrating Domain Knowledge and Reinforcement Learning. IEEE Transactions on Neural Networks and Learning Systems, Vol. 26, No. 5 (2015) 889-902.

[WT15] Di Wang and Ah-Hwee Tan. Creating Autonomous Adaptive Agents in a Real-Time First-Person Shooter Computer Game. IEEE Transactions on Computational Intelligence and AI in Games, Vol. 7, No. 2 (2015) 123-138.

[XT07]  Dan Xiao and Ah-Hwee Tan. Self-Organizing Neural Architectures and Cooperative Learning in Multi-Agent Environment. IEEE Transactions on Systems, Man and Cybernetics - Part B, Vol. 37, No. 6 (December 2007) 1567-1580.

[XT13]  Dan Xiao and Ah Hwee Tan. Cooperative Reinforcement Learning in Topology-based Multi-Agent Systems. Journal of Autonomous Agents and Multi-Agent Systems, Vol. 26, No. 1 (January 2013) 86-119 (DOI:10.1007/s10458-011-9183-4).

Contributions to Multimodal Information Fusion and Analysis

Information is power. The ability to access and use information holds the key to success in many strategic domains.  However, a large proportion of information today is in free-form text (just like this document). This motivated my research work on internet content mining, document categorization/clustering, personalization, and knowledge discovery from text.

Specifically, as the principal investigator of three projects, I pioneered the development of several text mining techniques that address the issues of making sense out of continual and large–scale text document inputs in a real-time incremental manner.

Our work on text classification has focused on the automatic classification of free-form documents according to user-defined categories using machine learning methods in an on-line and incremental manner. Using Adaptive Resonance Theory model that integrates both machine learning and domain knowledge, we are able to build document classifiers in an incremental manner, with a natural transition from human-directed operation to automatic routing. Traditional information organizing techniques can be either supervised (classification) or unsupervised (clustering). To combine the complementary strengths of the two paradigms, I developed a self-organizing model that provided the functions of both classification and clustering. Given a set of text documents, the system is able to discover groupings (clusters) through unsupervised learning. However, when user preference is given, the documents can be classified according to the user-defined categories (as in a supervised learning system) [TOP04]. The patented technique called user-configurable clustering forms the core of our business/competitive intelligence SDK software.

Information in the ubiquitous media age is typically fragmented and appears in various unstructured and unlabelled forms as data, text, image, audio, and video. In a joint project with the AStar Institute for Infocomm Research (I2R), we developed a framework and the critical technologies for analyzing, organizing, and delivery of mixed media information. As domain knowledge is the basis of media analysis and service delivery, we developed a suite of techniques for learning domain ontologies automatically from a given set of document collection [JT09b]. We further developed algorithms for learning user ontology based on a specialization of domain ontology [JT10].  On the challenging problem of media understanding, I introduced the use of domain ontology for bridging the semantic gap between raw image features and high level concepts. For integrating multimedia data, I proposed to develop a fusion model that associates visual features of images and keyword features of text annotation and performs cross-media associative pattern recall [JT09a]. This project has contributed to the Singapore’s key thrust in Interactive and Digital Media (IDM), through the development of technologies for automatic analysis of media data and production of digital media content. The key technologies developed in this project have been integrated into a showcase prototype, known as Media Workbench. In recent work, we further extended the model of Adaptive Resonance Theory to incorporate self-scaling vigilance control and applied it to social media data analysis [MTW16].

Selected Publications

[JT09a] Tao Jiang and Ah-Hwee Tan. Learning Image-Text Associations. IEEE Transactions on Knowledge and Data Engineering, Vol. 21, No. 2 (February 2009), 161-177.

[JT09b] Xing Jiang and Ah-Hwee Tan. Learning and Inferencing in User Ontology for Personalized Semantic Web Search. Information Sciences, Volume 179, Issue 16 (July 2009) 2794-2808. doi:10.1016/j.ins.2009.04.005

[JT10]  Xing Jiang and Ah-Hwee Tan. CRCTOL: A Semantic Based Domain Ontology Learning System. Journal of the American Society for Information Science and Technology, Vol. 61, No. 1 (January 2010), 150-168.

[TOP04] Ah-Hwee Tan, Hwee-Leng Ong, Hong Pan, Jamie Ng and Qiu-Xiang Li. Towards Personalized Web Intelligence. Knowledge and Information Systems, Vol. 6, No. 5 (September 2004) 595-616.

[TTW07] Tao Jiang, Ah-Hwee Tan and Ke Wang. Mining Generalized Associations of Semantic Relations from Textual Web Content. IEEE Transactions on Knowledge and Data Engineering, Vol. 19, No. 2 (February 2007) 164-179.

[MTW16] Lei Meng, Ah-Hwee Tan and Donald Wunsch, Adaptive Scaling of Cluster Boundaries for Large-scale Social Media Data Clustering, IEEE Transactions on Neural Networks and Learning Systems, Vol. 27, No. 12 (2016) 2656-2669.

Contributions to Biologically-Inspired Cognitive Architectures

Although many cognitive models have been proposed over the years, most of them rely on high level specifications and are not designed to learn and function continuously in real time [QTN09, DTF12]. Under a long-term collaboration with the Fusion and Cognition Lab of DSO National Laboratories since 2007, we embarked on a series of projects to develop biologically inspired integrated cognitive architectures by tapping the know-how in the fields of neurobiology and cognitive science. Specifically, I proposed an integrated self-organizing neural architecture, known as FALCON-X, which provides a unified framework for integrating a core set of high level cognitive capabilities, namely awareness, declarative knowledge representation, and reinforcement learning [TN11].

Within the overall architectural framework, we conducted in-depth study into a co-evolution theory for the representation, learning, and processing of human declarative memory, comprising the episodic and semantic memory. By taking an interdisciplinary approach, we investigated the process of episodic memory formation [WST12] and how such episodic representation may be translated into the more structured and permanent semantic memory store [SWT12]. Working from the other way round, we also studied and modeled the role of semantic memory in making sense of one’s experience and guiding decision making in procedural memory [WST17].

Selected Publications

[DTF12] Włodzisław Duch, Ah-Hwee Tan and San Franklin. Cognitive architectures and autonomy: Commentary and Response. Journal of Artificial General Intelligence, Volume 3, Issue 2, Pages 31–63, ISSN (Online) 1946-0163, November 2012.

[QTN09] Hui-Qing Chong, Ah-Hwee Tan and Gee-Wah Ng. Integrated Cognitive Architectures: A Survey. Artificial Intelligence Review, Vol. 28, No. 2 (Published Online February 2009) 103-130. doi:10.1007/s10462-009-9094-9

[SWT12] Budhitama Subagdja, Wenwen Wang, Ah-Hwee Tan, Yuan-Sin Tan, and Loo-Nin Teow. Memory Formation, Consolidation, and Forgetting in Learning Agents. In Proceedings, Eleventh International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2012), pp. 1007-1014, Valencia, Spain, June 4-8, 2012.

[TN10]  Ah-Hwee Tan and Gee-Wah Ng. A Biologically-Inspired Cognitive Agent Model Integrating Declarative Knowledge and Reinforcement Learning. In proceedings, IEEE/WIC/ACM International Conference on Intelligent Agent Technology, pp. 248-251, Toronto, August 31 - September 3, 2010.

[TTT12] Teck-Hou Teng, Ah-Hwee Tan, Yuan-Sin Tan and Adrian Yeo. Self-organizing neural networks for learning air combat maneuvers. In Proceedings, International Joint Conference on Neural Networks, pp. 2859-2866, Brisbane, Australia, June 10-15, 2012.

[WST09]Di Wang, Budhitama Subagdja, Ah-Hwee Tan and Gee-Wah Ng. Creating Human-like Autonomous Players in Real-time First Person Shooter Computer Games. In proceedings, Twenty-First Annual Conference on Innovative Applications of Artificial Intelligence (IAAI'09), Pasadena, California, July 14–16, 2009.

[WST12] Wenwen Wang, Budhitama Subagdja, Ah-Hwee Tan and Janusz A. Starzyk. Neural Modeling of Episodic Memory: Encoding, Retrieval, and Forgetting. IEEE Transactions on Neural Networks and Learning Systems, Vol. 23, No. 10 (October 2012), 1574–1586.

[WST17] Wenwen Wang and Ah-Hwee Tan. Semantic Memory Modelling and Memory Interaction in Learning Agents. IEEE Transactions on Systems, Man and Cybernetics: Systems, Vol. 47, No. 12 (2017) 2882-2895.

Contributions to Human-Like Characters in Virtual Environment

Supported by the Singapore National Research Foundation - Interactive Digital Media (NRF-IDM) Programme, I led a research team in developing autonomous virtual agents for 3D interactive environment called Co-Space. Co-Space is an initiative championed by MDA to develop interactive virtual worlds that are typically modeled after a real physical environment in terms of content, look-and-feel and functionalities. To enrich the experience of individual users in Co-Space, it is critical to incorporate knowledge and intelligent facilities to enhance the interactivity and playability within. In this project, we developed and populated human-like cognitive agents in the form of autonomous avatars, who roam in the landscape of Co-Space, gather awareness of its surrounding and interact with users through their human avatars. With the autonomous avatars befriending and providing personalized context-aware services to human avatars, the content and services in Co-Space can then become active and readily available to the users.

Based on fusion Adaptive Resonance Theory (ART), I have developed an integrated cognitive agent model based on which the autonomous avatars can perform a myriad of cognitive functions, including recognition, prediction and learning [ST12; TFO10; TOT11]. Following the blueprint of NTU campus, we developed a 3D interactive virtual environment called NTU Co-Space as the development and evaluation platform for autonomous avatars. The NTU Co-Space features a mini-quest that provides the virtual experience for a visitor to see and experience the environment of the NTU campus through a fun and interactive journey [KTN12]. The NTU Co-Space with the amazing quest was showcased in the NTU Freshman Welcome Week in August 2011 and August 2012. It was also presented at DemoAsia 2012, the demonstration track of the AAMAS’2012 [KST12], as well as at ICIS’2012, the flagship conference in the field of Information Systems [KTN12].

Selected Publications

[KST12] Yilin Kang, Budhitama Subagdja, Ah-Hwee Tan, Yew-Soon Ong and Chun-Yan Miao. Virtual Characters in Agent-Augmented Co-Space. In Proceedings, Eleventh International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2012) (Demo Track), pp. 1465-1466, Valencia, Spain, June 4-8, 2012.

[KNT12] Yilin Kang, Fui-Hoon Nah and Ah-Hwee Tan. Investigating Intelligent Agents in a 3D Virtual World. In Proceedings, International Conference on Information Systems (ICIS 2012), Orlando, USA, December 16-19, 2012.

[KTN12] Yilin Kang, Ah-Hwee Tan and Fui-Hoon Nah. Agent-based Virtual Humans in Co-Space: An Evaluative Study. In Proceedings, IEEE/WIC/ACM International Conference on Intelligent Agent Technology (IAT 2012), Macau, China, December 4-7, 2012.

[ST12]  Budhitama Subagdja and Ah-Hwee Tan. iFALCON: A Neural Architecture for Hierarchical Planning. Neurocomputing, Vol. 86 (June 2012) 124-139 (DOI:10.1016/j.neucom.2012.01.008).

[TFO10] Ah-Hwee Tan, Yuhong Feng and Yew-Soon Ong. A Self-Organizing Neural Architecture Integrating Desire, Intension and Reinforcement Learning. Neurocomputing, Vol. 73, No. 7-9. (March 2010), 1465-1477.

[TOT11] Ah-Hwee Tan, Yew-Soon Ong and Akejariyawong Tapanuj. A Hybrid Agent Architecture Integrating Desire, Intention and Reinforcement Learning. Expert Systems with Applications, Volume 38, No. 7 (July 2011) 8477-8487.

Contributions to Assistive Agents for Aging In Place

One of my key focus areas is to develop cognitive based assistive technologies for supporting the active life style of the elderly. In collaboration with Miao Chunyan (PI) and Cyril Leung,

I led a research program on Aging in Place under the NTU-UBC Research Centre of Excellence in Active Living for the Elderly (LILY).

Aging in place is often preferred by elderly to live out their days independently and with dignity in the familiar comfort of their own homes. Considering that the elderly may be relatively immobile and living alone, it is imperative that caregiving help, both in physical and mental aspects, should be available to them at all times. With this motivation, this program aims to develop a suite of assistive technologies collectively called Silver Assistants for supporting living in place.

To serve the caregiving duties for the elderly, Silver Assistants need to be proactive, be a good listener, who can understand the needs and feeling of the user and provide necessary services under appropriate circumstances. To achieve the above, the key properties of Autonomy and Interactivity have been developed as highlighted below.

Autonomy Based on fusion Adaptive Resonance Theory (ART), we developed an integrated agent model based on which the Silver Assistants can perform a myriad of cognitive functions, including recognition, prediction and learning, in response to a continual stream of multi-modal input signals received from the user [WSKT15]. The Silver Assistants make decisions not only based on situational factors perceived from the environment but also the user’s mental states characterized by his/her autobiographic memory, desire and intention [TOT11; WTM16]. By modelling the internal states explicitly, the Silver Assistants can be more spontaneous and able to interact and learn in real time.

Interactivity For interaction between the Silver Assistants and the users, we developed an intuitive user interface, through which a user may request specific services. The user could also interact with his/her assistants through verbal expressions, either by text or voice. More importantly, the Silver Assistants build an internal model of the users, with his/her profile, interests and preferences, and daily activity patterns. The user model in turns allows the agent to make intelligent conversation and recommendations on topics relevant to the user [KTM15; GWTM15].

Selected Publications

[GWTM15] Shan Gao, Di Wang, Ah-Hwee Tan, and Chunyan Miao. Progressive Sequence Matching for ADL Plan Recommendation. In proceedings, IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT 2015), pp360-367, Singapore, December 6-9, 2015.

[KTM15] Yilin Kang, Ah-Hwee Tan, and Chunyan Miao. An Adaptive Computational Model for Personalized Persuasion. In proceedings, 14th International Conference on Artificial Intelligence (IJCAI 2015), Buenos Aires, pp61-67.

[TOT11] Ah-Hwee Tan, Yew-Soon Ong and Akejariyawong Tapanuj. A Hybrid Agent Architecture Integrating Desire, Intention and Reinforcement Learning. Expert Systems with Applications, Volume 38, No. 7 (July 2011) 8477-8487.

[WSKT15] Di Wang, Budhitama Subagdja, Yilin Kang, and Ah-Hwee Tan. Silver Assistants for Aging-in-Place. In proceedings, IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT 2015), pp229-230, Singapore, December 6-9, 2015.

[WTM16] Di Wang, Ah-Hwee Tan and Chunyan Miao. Modeling Autobiographical Memory in Human-Like Autonomous Agents. In Proceedings, 15th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2016), Singapore, May 9-13, 2016.