Easy Learning Adaptive Nimble
Digital neural network based smart devices


Project
Consortium
People
Members area

Introduction

In the computer era many technology applications are not user friendly and often require experts to be considered useful. The efforts to build more natural Human Computer Interfaces (HCI) seem to be not enough where the lack of adaptive capability requires, however, parameters setting activity and then knowledge.
The goal of this research project is to build a framework for developing smart devices based on the vectorial representation of reality, learning capability and adaptive behavior of digital neural networks.
Target applications are smart cameras for work environment safety, adaptive automotive drive assistants, intelligent computer periferals enabling people with any kind of handicap to use computer and communicate. In an other way, these devices could be integrated as transparent learning and adaptive controllers for any kind of domestic and industrial machinery.
The goal is the definition of a common Very High Speed Integrated Circuit Hardware Description Language (VHDL) framework enabling a fast development of specialized applications with the addition of tailored VHDL components and specific hardware sensors. This framework should be portable on many low cost FPGA development boards.
 

The framework has the following features:

 

Scientific and technological objectives of the project


The project main objective is the definition and design of a framework that could become a standard for the development of smart devices with the following characteristics:
Low cost:
The project is based on available FPGA boards that are normally produced by FPGA core-business companies and third part companies as development/evaluation platforms.
The baseline for the choice of the hardware platform must be the minimal configuration such as the board with one or more FPGA and a (E)PROM for automatic configuration of the FPGA at bootstrap. Anyway the architecture developed in the project is portable on any (note.1) FPGA based board considering that special components eventually built on board should be provided of common interface VHDL drivers for their class as well as external devices.
Also low cost boards have high speed buses and then can be connected in multiple instances distributing complex behaviours on different FPGA and different boards.
Independence from hardware platform:
The development framework that is the core of the project must not be related to any existing board with specific additional behaviours. The development framework must warranty that any application can work on any FPGA (note.1) board or FPGA board system (more boards connected).
Only the change of communication devices and sensors needs the development of new drivers while the main application and basic behaviour modules should not be modified.
Any module should be documented with resource requirements and possibly a list of actually available FPGAs in which it can be implemented.
Reusability of VHDL components:
These components have standard behaviour-dependent interfaces
These are semantic or application related VHDL modules that can perform specific and common sub-tasks (ex: features extraction). These must have a «server» role for the application. These components must have a specific common interface well defined for any kind of specific behaviour (ex: composite profile extraction). The UML «Logical view diagram» shows the concepts of common interface for specific functionality that enables the reusability of components.
Exchangeability of sensors, I/O devices and external memory:
VHDL drivers for sensors, communication ports, and memory must have a defined interface for any specific function/device type (ex: colour CCD camera) and must work as servers.
The UML «Logical view diagram» shows the concept of common interfaces of drivers for specific classes of devices that enables the exchangeability of devices in the same class (see also UML <<Deployment view diagram>>).

(note.1) constraints should be related only to resources (size and architecture dependent) capacity of FPGA and resources requirements of behavioural VHDL modules
 

Learning and adaptive behaviour:
This is concentrated in one or more VHDL Neural Network modules. These new NN paradigms tailored for hardware implementation should be developed inside the project.
These VHDL modules are neural network algorithms tailored for hardware implementation and are the key for the adaptive and learning behaviour of the development framework. Any application of a new smart device should use these «neural engines» to add adaptive and learning capability.
These modules, as any other basic behaviour module in the architecture, work as servers for the application.
An example of these NN is a Radial Basis Function NN tailored for hardware implementation.
Typical Radial Basis Function implementations in hardware have high CLB (Configurable Logic Blocks) consumption and then few neurones (few prototypes) can be realised on actual FPGA.
This paradigm has the same basic behaviour and advantages of RBF NN:

But this NN can be realised in FPGA with high efficiency prototypes/CLB ratio and works as a «data-driven» device.
This paradigm is able to grow in the number of neurones without limiting performance and requiring only additional external memory.
Without need of hardware design and fabrication:
This framework is based on existing low cost FPGA boards. These can be connected in multiple instances with fast (up to 100MHz) communication channels and connected also to external I/O devices and memory modules.
These boards are normally produced, for evaluation and development of OEM devices, by the same FPGA fabricators and third part companies.
Avoiding design and fabrication of hardware, smart devices can be produced concentrating any effort on development of the application. Furthermore very small companies with very low fabrication resources can produce these devices. The following UML deployment view diagram shows the philosophy of hardware deployment of the framework.

Fast time to market:
The target is a framework with hardware modules that can be easily connected and VHDL modules that can manage sensors, basic behaviours (ex: features extraction, pattern recognition), external memory modules. Having this framework, the development of a new smart device needs only design and synthesis of an application specific VHDL module. This framework must define clearly the interfacing rules of modules enabling people to develop new drivers for new sensors and new implementations of basic behaviour modules.
Development of new basic behaviours or new classes device-drivers needs the definition of interface specifications compliant with the generic interfacing rules of the framework.
An example could be development of drivers for linear cameras and stereovision features extraction modules. In this way the framework would be growing anytime it would be used in a new application field.
 
 

Project steps:

The project is developed in the following basic steps:
Platform analysis and design
Platform components development and test
Example target application(s) analysis and design
Example target application(s) development and test
 
PROJECT TASKS:
FHD = Framework Hierarchy Definition
CIRD = Common Interface Rules Definition
SIRD = Specific Interface Rules Definition
AAD = Application Analysis and Design
CBD = Common Behaviors Design
DDD = Device Drivers Design
NND = Neural Networks Design
VDT = VHDL Development and Test
SWD = Software Design and Development 

Platform analysis and design:

Platform components development: Example target application(s) analysis and design: Example target application(s) development and test:

State of the art


Due to the interdisciplinarity of the project it is not easy identify a unique «state of the art».
We can identify a «state of the art» in the following fields related with the targets of the project:

Probably industries working in hardware design have developed their own frameworks in order to accelerate the process of design for customers and actually many VHDL components can be bought for specific tasks and many are free and can be downloaded from Internet but a recognised library with an organised hierarchy of functionality is not available. A framework with these features has been never realised or made available and never a standard has been created. Probably theory about the concept of «smart devices» have been explored and rules to build adaptive and learning devices have been already written in some feasibility studies but never a development framework (both for hardware and software) has been designed to match these requirements. Hardware implementations of neural networks have been few and not successful because of the following reasons:
- missing focus on applications
- optimisation not starting from the paradigm but only from the implementation
- poor interaction and knowledge exchange between hardware design engineers and neural scientists
The ELAN project has the objective and all the instruments to fill these lacks.
 
 

Potential impact

A three levels industrial exploitation is expected:

1) Industrial standard
2) VHDL library
3) FPGA based devices

End-user should be in one of the following categories:

1) small companies having know-how to design software and hardware but having not resources to produce hardware.  In the last years,  a lot of small software-houses have been able to offer competitive solutions in a market where bigger companies where consolidated. This fact didn’t happen for hardware because the production cost (added anyway to design cost) is a big obstacle for small companies to be competitive in this field. The framework should enable small companies to produce electronic devices


2) big companies that have enough resources for the production of  hardware but consider a flexible and «fast time to market» solution more adequate for devices characterised by rapid and continuous evolution for growing performances or changing features demand.

Generally, considering the «smart devices» philosophy of the framework (such as devices where intelligence is embedded and hidden at user), those companies should operate principally on automotive, «domotics» and automation. In the first two fields the trend is production of easy-to-use devices with a philosophy completely different from that of personal computer.

In the past years we have seen many small software-house companies realise products that have been competitive with products from big companies. This situation has been possible due to the fact products based only on software requires knowledge and not high investments in hardware resources and instruments. So we have assisted at the birth of many small companies in this field.
In other way we have never seen small companies produce hardware products competitive in the market. The main problem is that hardware production requires investments that goes over the human resources and often some activities are realised in outsourcing. This situation is critical for small companies and often stops their activity in the first years of life.
The rules that enable a small company to survive and be competitive in the market of hardware electronic products are substantially two:

The results of the project are directly tuned to realise the two above conditions.
The market of «smart devices» intended as devices that can work with an «hidden intelligence», is growing in demand (automotive, domotics, monitoring) and actually not many companies have knowledge to produce them. There is now space for a new generation of small knowledge-based companies that could be competitive in this field.
The role of a framework and possibly an industrial standard would be fundamental to enable these small companies to develop hardware devices corresponding to the features described above, at low cost.
The major societal and economical impact should be related with the creation of a new high technology market and a new opportunity for entrepreneurs: this is the reason we consider an added-value in carrying out the work at a European level.
Enabling small companies to produce intelligent devices for domotics, automotive and automation would have a strong positive effect in these markets. While many companies are working in industrial automation, the situation is different for the automotive field where few big companies have monopoly of market. The introduction of SMEs in this market should help the evolution and accelerate the enhancement process of  electronic devices for automotive applications. The possible downgrade of prices influenced also by a larger competition should determine an enlargement of the application area: sophisticated devices could be applied on low cost cars.
The domotics market is just a relative new market and it is probably the most fertile terrain for the application of smart devices. We have to include in this market also those devices that could help people with handicap to interact with home environment: low cost voice recognition devices could be embedded in any electrical appliance. Smart devices could be a human interface for people with handicap also for communication purpose and enable them to use efficiently personal computers and connect to the Internet: speak to text, text to speak devices and facial expression recognition devices could substitute keyboard, monitor and mouse. In this way an «e-inclusion» market could be started and competition with low cost of hardware production should be the key to have affordable prices. Having the instruments to build these devices without hardware production and using general purpose hardware devices (FPGA boards), a small company could decide to specialise in this field with low investment risk.
 
 

Contributions to standards


VHDL is a standard (VHDL-1076) developed by the IEEE. The language has been through a couple of revisions, the most widely used version is the 1987 (Std 1076-1987) version, sometimes referred to as VHDL'87, but also just VHDL. However, there is a newer revision of the language referred to as VHDL'93. VHDL'93 (adopted in 1994 of course) is fairly new and is still in the process of replacing VHDL'87.
The project uses the VHDL standard language and wants to produce a standard framework on top of this language. Really the framework should have rules that enable the implementation virtually in any kind of hardware description language but the real implementation of the framework (the library) is realised using VHDL standard.
The main goal of the project is the creation of an international standard for the design of smart devices that enables reuse of components and exchangeability of hardware reducing production costs.
Actually a generic standard that defines common interfaces for VHDL modules related to specific functionality is missing and normally modules implementing the same functionality have completely different interfaces. The project is oriented to the development of smart devices based on cognitive and neural behaviours but the framework functionality hierarchy is really open to the addition of any new directory related to a different field (as example telecommunication).
A non-monolithic management of the framework library and granularity of IP should help partners of the consortium to apply the work developed inside the project on their own typical market. In this way the project strengthens its vocation to create a «standard» for fast time-to-market development of smart devices instead of a simple VHDL library.