Spatial Reasoning: From Sketch-to-Text towards Text-to-Sketch


James M. Keller
University of Missouri, USA


With the collaboration of several faculty colleagues and many students, I have been studying the creation and utilization of spatial relations in various sensor-related domains for many years. Scene description, involving linguistic expressions of the spatial relationships between image objects, is a major goal of high-level computer vision. In a series of papers, we introduced the use of histograms of forces to produce evidence for the description of relative position of objects in a digital image. There is a parameterized family of such histograms, for example, the histogram of constant forces (much like the earlier histogram of angles) and the histogram of gravitational forces that highlights areas that are close between the two objects. Utilizing the fuzzy directional membership information extracted from these histograms within fuzzy logic rule-based systems, we have produced high-level linguistic descriptions of natural scenes as viewed by an external observer. Additionally, we have exploited the theoretical properties of the histograms to match images that may be the same scene viewed under different pose conditions. In fact, we can even recover estimates of the pose parameters. These linguistic descriptions have then been brought into an ego-centered viewpoint for application to robotics, i.e., the production of linguistic scene description from a mobile robot standpoint, spatial language for human/robot communication and navigation, and understanding of a sketched route map for communicating navigation routes to robots. This last activity is sketch-to-text. In a newly awarded grant from the National Geospatial Intelligence Agency, we are starting to tackle the inverse problem: given one or more text descriptions of a temporal and spatial event, construct a sketch of the event for subsequent reasoning. The sketch must be grounded in reality by matching to a satellite image or geospatial database. This talk will survey the early applications and end with a demo highlighting our approach to the new problem.


James M. Keller received the Ph.D. in Mathematics in 1978. He holds the University of Missouri Curators¡¯ Professorship in the Electrical and Computer Engineering and Computer Science Departments on the Columbia campus. He is also the R. L. Tatum Professor in the College of Engineering. His research interests center on computational intelligence: fuzzy set theory and fuzzy logic, neural networks, and evolutionary computation with a focus on problems in computer vision, pattern recognition, and information fusion including bioinformatics, spatial reasoning in robotics, geospatial intelligence, sensor and information analysis in technology for eldercare, and landmine detection.

His industrial and government funding sources include the Electronics and Space Corporation, Union Electric, Geo-Centers, National Science Foundation, the Administration on Aging, The National Institutes of Health, NASA/JSC, the Air Force Office of Scientific Research, the Army Research Office, the Office of Naval Research, the National Geospatial Intelligence Agency, and the Army Night Vision and Electronic Sensors Directorate. Professor Keller has coauthored over 300 technical publications. Jim is a Fellow of the Institute of Electrical and Electronics Engineers (IEEE) for whom he has presented live and video tutorials on fuzzy logic in computer vision, is an International Fuzzy Systems Association (IFSA) Fellow, is a national lecturer for the Association for Computing Machinery (ACM), is an IEEE Computational Intelligence Society Distinguished Lecturer, and is a past President of the North American Fuzzy Information Processing Society (NAFIPS). He received the 2007 Fuzzy Systems Pioneer Award from the IEEE Computational Intelligence Society. He finished a full six year term as Editor-in-Chief of the IEEE Transactions on Fuzzy Systems, is an Associate Editor of the International Journal of Approximate Reasoning, and is on the editorial board of Pattern Analysis and Applications, Fuzzy Sets and Systems, International Journal of Fuzzy Systems,and the Journal of Intelligent and Fuzzy Systems. He was the Vice President for Publications of the IEEE Computational Intelligence Society from 2005-2008, and is currently an elected Adcom member. He was the conference chair of the 1991 NAFIPS Workshop, program co-chair of the 1996 NAFIPS meeting, program co-chair of the 1997 IEEE International Conference on Neural Networks, and the program chair of the 1998 IEEE International Conference on Fuzzy Systems. He was the general chair for the 2003 IEEE International Conference on Fuzzy Systems.

Statistics Powered Conceptual Fuzzy Sets and Word Computing System

Tomohiro Takagi
Meiji University, Japan

According to Gorge Lakoff abstract concepts are acquired as generalization results of experimental similarities and experimental co-occurrences. According to the use theory of meaning by Wittgenstein, the meaning of a word is defined when it is used and can be expressed by other words. Regarding both points, we have proposed conceptual fuzzy sets (CFS henceforth) as augmented fuzzy sets as a system acquiring and representing context sensitive word meanings. Ordinary fuzzy sets are defined with elements with their membership values. In the CFS, the meaning of a word is expressed by the set of relative words and their relationship values.
The strongest cause of word ambiguity is context. As the meaning of a word is not fixed and changes dynamically depending on context which varies depending on specificity of the topic and the viewpoint of a writer. Ordinary fuzzy sets express phenomena with an un-clear boundary. However, when applying them to various realistic matters, problems occur because the ambiguity of the fuzzy sets is fixed. We proposed two methodologies to cope with above problem by representing context dependent word meaning by changing the figures of the CFS reflecting the contexts. In the first method, we generate a conceptual fuzzy set by the superpositioning of prototype concepts, which are obtained as clusters of words. The meaning varying depending on the context can be generated by superpositioning the prototype concepts with similarities to the input. By conceptual matching using this meaning representation method, we won 1st place at ImageCLEF photo retrieval task EN_AUTO_TXT, where participants retrieve a photo only with annotated text description, last year 2008. The second one computes relationship values between words as mutual information. We used a large corpus consisting of 1 million newswire text data in our experiments. We will demonstrate that our method effectively works not only representing word meanings and but obtaining appropriate granularities showing experimental results.
Using the CFS mentioned above we are challenging to build a different word computing system from the CWW system proposed by Prof. Zadeh. When ¡°A implies B¡± and A¡¯ are given in ordinary fuzzy logic, we obtain B¡¯ as a result of approximate reasoning using similarity between A and A¡¯. Actually however similarity is not a simple idea. For example, ¡°natural gas¡± is different from ¡°oil¡± viewing as materials, because the former is gaseous and the latter is liquid. But from a view point of combustibility, their similarity is very high. This example shows viewpoints affect degrees of similarity seriously, which is called metaphor-induced similarity in cognitive linguistics. Thus, building general approximate reasoning, we ought to incorporate functions to deal with the general similarities. Such a reasoning process is illustrated in the following 4-terms reasoning.

-------- car: gasoline = sailboat : x?

We human usually reason "wind" as this x. However ordinary fuzzy reasoning cannot obtain ¡°wind¡± as a result. To achieve this reasoning the relation of ¡°car¡± and ¡°gasoline¡± should be once interpreted in an abstract notion and then x should be reasoned from ¡°sailboat.¡± An approximate reasoning method using CFS enables ¡°wind¡± to be reasoned. Here I show the method and experiment results. Now we challenge to apply the above methods to information recommender system in accordance with user profile and economic fluctuation prediction referring past fluctuations.

Tomohiro Takagi received the Doctor of Engineering degree from the Tokyo Institute of Technology in 1983. He was an EECS research fellow at the University of California Berkeley from 1983-84. From 1987 to 1997 he worked in the central research laboratory and corporate multimedia promotion division at Matsushita Electric Industrial Co., LTD. He was also a deputy director at the Laboratory for International Fuzzy Engineering Research, which was a national project supported by the Ministry of International Trade and Industry, from 1991 to 1993. Since 1998 he has been affiliated with the Department of Computer Science of Meiji University.

Fuzzy-based Learning of Human Behavior Patterns

Z. Zenn Bien
KAIST, Korea

In designing autonomous service systems in a smart home environment, discovery and prediction of human behaviors are often crucial. For patterns of human behavior with inherent ambiguity and uncertainty, however, their modeling and recognition are known a challenging task. In this talk are presented some effective fuzzy model-based learning techniques, which our research group has devised. These include, in particular, an interpretable probabilistic fuzzy rule base, a non-supervised fuzzy Q-learning, and an application study of human behavior suggestion system for memory impaired people. Efficient learning schemes with less human intervention and less prior knowledge are expected to be developed in this direction of research.

Zeungnam ¡®Zenn¡¯ Bien received his B.S. degree in Electronics Engineering from Seoul National University, Seoul, Korea, in 1969 and, M.S. and Ph.D. degrees in electrical engineering from the University of Iowa, Iowa City, Iowa, U.S.A., in 1972 and 1975, respectively. During 1976-1977 academic years, he taught at Department of Electrical Engineering, University of Iowa. Then Dr. Bien joined Korea Advanced Institute of Science and Technology, summer, 1977, and had worked for Department of Electrical Engineering & Computer Science, KAIST as professor till February 28, 2009. He has become a Chaired Professor of UNIST (Ulsan National Institute of Since and Technology) since March 1, 2009.

He was a visiting faculty at the University of Iowa for a year since Sep. 1981, and a visiting researcher at CASE Center of Syracuse University, New York, while a visiting professor at Department of Control Engineering, Tokyo Institute of Technology during 1987-1988 academic years. He had been in INT, France and TDU, Japan as a visiting professor from Sept. 1, 2006 till August 31, 2007, six month each for his sabbatical year. Prof. Bien has been serving for a number of professional societies, domestic and overseas. He was the founding president of the Korea Fuzzy Logic and Intelligent Systems Society during 1990-1995 and also, the general chairs for IFSA World Congress 1993, and for FUZZ-IEEE99, respectively. Dr. Bien served as the President of the Institute of Electronics Engineers of Korea (IEEK) for the year 2001. He worked as the president for International Fuzzy Systems Association (IFSA) during 2003-2005. For Korea Academy of Science and Technology, he served as the chairman of Engineering Division for 3 years and is now a member of Board of Directors. Dr. Bien was also the founding president of Korea Robotics Society for 2003~6. At KAIST, Prof. Bien served as Dean of Academic Affairs, Dean of College of Engineering and Director of Human-friendly Welfare Robot System Engineering Research Center for 9 years since 1999. Being a KEPCO-Chaired Professor during 2005~2009, he is now a Professor Emeritus of KAIST. He served as an editorial advisory board member for International Journal of Fuzzy Systems (IJFS), an editorial board member for IEEE Transactions on Fuzzy Systems, and an associate editor for Fuzzy Optimization and Decision Making (FODM). Prof. Bien was the Editor-in-chief of International Journal of Assistive Robotics and Mechatronics. Prof. Bien has been awarded a number of domestic and international prizes including National Hyokshin medal, World Automation Congress Award, Joshep Engelberger Award and many others. He is an IEEE Fellow, also an IFSA FUZZY Fellow and members of KAST and NAEK. His current research interests include Intelligent System and Control with particular attention to Assistive Service Robotic Systems and Smart Homes-Intelligent Village. Prof. Bien has published more than 460 international journal/proceedings papers, and has authored/coauthored 7 technical books. He has obtained 22 patents registered and 5 pending.