Easy
Learning Adaptive
Nimble
Digital neural
network based smart devices
Introduction
In the computer era many technology applications are
not user friendly and often require experts to be considered useful. The
efforts to build more natural Human Computer Interfaces (HCI) seem to be
not enough where the lack of adaptive capability requires, however, parameters
setting activity and then knowledge.
The goal of this research project is to build
a framework for developing smart devices based on the vectorial representation
of reality, learning capability and adaptive behavior of digital neural
networks.
Target applications are smart cameras for work
environment safety, adaptive automotive drive assistants, intelligent computer
periferals enabling people with any kind of handicap to use computer and
communicate. In an other way, these devices could be integrated as transparent
learning and adaptive controllers for any kind of domestic and industrial
machinery.
The goal is the definition of a common Very High
Speed Integrated Circuit Hardware Description Language (VHDL) framework
enabling a fast development of specialized applications with the addition
of tailored VHDL components and specific hardware sensors. This framework
should be portable on many low cost FPGA development boards.
The framework has the following features:
-
Low cost: based on general purpose FPGA
boards without specific components and interfaces. These low cost boards
are even available in the market for development and FPGA evaluation purpose.
-
Independence from hardware platform: portable
on many FPGA boards.
-
Reusability of VHDL components with common
functionality-dependent interfaces.
-
Exchangeability of sensors, I/O devices
and external memory modules thanks to common device class VHDL drivers
interfaces.
-
Learning and adaptive behaviour concentrated
in one or more VHDL Neural Network modules. These new NN paradigms tailored
for hardware implementation should be developed inside the project.
-
No need of hardware design and fabrication.
-
Fast time to market: development of new
smart devices needs only design of new application-specific VHDL modules.
-
Modularity: more complex systems can be
created connecting many FPGA boards.
Scientific and technological objectives of the
project
The project main objective is the definition
and design of a framework that could become a standard for the development
of smart devices with the following characteristics:
Low cost:
The project is based on available FPGA boards
that are normally produced by FPGA core-business companies and third part
companies as development/evaluation platforms.
The baseline for the choice of the hardware platform
must be the minimal configuration such as the board with one or more FPGA
and a (E)PROM for automatic configuration of the FPGA at bootstrap. Anyway
the architecture developed in the project is portable on any (note.1) FPGA
based board considering that special components eventually built on board
should be provided of common interface VHDL drivers for their class as
well as external devices.
Also low cost boards have high speed buses and
then can be connected in multiple instances distributing complex behaviours
on different FPGA and different boards.
Independence from hardware platform:
The development framework that is the core of
the project must not be related to any existing board with specific additional
behaviours. The development framework must warranty that any application
can work on any FPGA (note.1) board or FPGA board system (more boards connected).
Only the change of communication devices and sensors
needs the development of new drivers while the main application and basic
behaviour modules should not be modified.
Any module should be documented with resource
requirements and possibly a list of actually available FPGAs in which it
can be implemented.
Reusability of VHDL components:
These components have standard behaviour-dependent
interfaces
These are semantic or application related VHDL
modules that can perform specific and common sub-tasks (ex: features extraction).
These must have a «server» role for the application. These
components must have a specific common interface well defined for any kind
of specific behaviour (ex: composite profile extraction). The UML
«Logical view diagram» shows the concepts of common
interface for specific functionality that enables the reusability of components.
Exchangeability of sensors, I/O devices and external
memory:
VHDL drivers for sensors, communication ports,
and memory must have a defined interface for any specific function/device
type (ex: colour CCD camera) and must work as servers.
The UML «Logical
view diagram» shows the concept of common interfaces of drivers
for specific classes of devices that enables the exchangeability of devices
in the same class (see also UML <<Deployment
view diagram>>).
(note.1) constraints should be related only to
resources (size and architecture dependent) capacity of FPGA and resources
requirements of behavioural VHDL modules
Learning and adaptive behaviour:
This is concentrated in one or more VHDL Neural
Network modules. These new NN paradigms tailored for hardware implementation
should be developed inside the project.
These VHDL modules are neural network algorithms
tailored for hardware implementation and are the key for the adaptive and
learning behaviour of the development framework. Any application of a new
smart device should use these «neural engines» to add adaptive
and learning capability.
These modules, as any other basic behaviour module
in the architecture, work as servers for the application.
An example of these NN is a Radial Basis Function
NN tailored for hardware implementation.
Typical Radial Basis Function implementations
in hardware have high CLB (Configurable Logic Blocks) consumption and then
few neurones (few prototypes) can be realised on actual FPGA.
This paradigm has the same basic behaviour and
advantages of RBF NN:
-
New learning doesn’t affect previous learning
-
Transparent behaviour (doesn’t work as a black-box)
-
Computational simplicity
But this NN can be realised in FPGA with high efficiency
prototypes/CLB ratio and works as a «data-driven» device.
This paradigm is able to grow in the number of
neurones without limiting performance and requiring only additional external
memory.
Without need of hardware design and fabrication:
This framework is based on existing low cost FPGA
boards. These can be connected in multiple instances with fast (up to 100MHz)
communication channels and connected also to external I/O devices and memory
modules.
These boards are normally produced, for evaluation
and development of OEM devices, by the same FPGA fabricators and third
part companies.
Avoiding design and fabrication of hardware, smart
devices can be produced concentrating any effort on development of the
application. Furthermore very small companies with very low fabrication
resources can produce these devices. The following UML deployment view
diagram shows the philosophy of hardware deployment of the framework.
Fast time to market:
The target is a framework with hardware modules
that can be easily connected and VHDL modules that can manage sensors,
basic behaviours (ex: features extraction, pattern recognition), external
memory modules. Having this framework, the development of a new smart device
needs only design and synthesis of an application specific VHDL module.
This framework must define clearly the interfacing rules of modules enabling
people to develop new drivers for new sensors and new implementations of
basic behaviour modules.
Development of new basic behaviours or new classes
device-drivers needs the definition of interface specifications compliant
with the generic interfacing rules of the framework.
An example could be development of drivers for
linear cameras and stereovision features extraction modules. In this way
the framework would be growing anytime it would be used in a new application
field.
Project steps:
The project is developed in the following basic steps:
Platform analysis and design
Platform components development and test
Example target application(s) analysis and design
Example target application(s) development and
test
| PROJECT TASKS: |
FHD = Framework Hierarchy
Definition
CIRD = Common Interface
Rules Definition
SIRD = Specific Interface
Rules Definition
AAD = Application Analysis
and Design
CBD = Common Behaviors
Design
DDD = Device Drivers
Design
NND = Neural Networks
Design
VDT = VHDL Development
and Test
SWD = Software Design
and Development |
Platform analysis and design:
-
Definition of system requirements
-
Definition of minimal hardware requirements
-
Identification of low cost commercially available
FPGA boards from different companies matching the requirements
-
Analysis and design of system and it’s components
hierarchy
-
Definition of standard interfaces for VHDL device
class drivers
-
Definition of standard interfaces and design of basic
behaviour modules
-
Analysis and design of NN paradigms tailored for VHDL
implementation with minimal resources consumption and definition of standard
interfaces for any NN class
Platform components development:
-
Identification of specific commercially available
sensors
-
Development of VHDL class drivers for specific sensors
following the specifications that are output of the analysis and design
step
-
Development of VHDL basic behaviours modules following
the specifications that are output of the analysis and design step
-
Development of VHDL Neural Networks following the
specifications that are output of the analysis and design step
-
Simulation and test of all VHDL modules
Example target application(s) analysis and design:
-
Identification of a specific target device performing
a goal-directed task
-
Analysis of the application behaviour and identification
of specific sensors and I/O requirements, basic behaviours requirements
and neural network typology requirements
-
Design of the application behaviour using the Neural
Network paradigm, basic behaviour modules and drivers
-
Analysis and design of special behavioural requirements
as learning facility with connection of the device to a personal computer
-
Simulation and test on personal computer of the system
Example target application(s) development and
test:
-
Development of the VHDL application block implementing
the behaviour as described in the specifications that are output of the
design step
-
Simulation and test on personal computer of the VHDL
application block
-
Development of software for learning facility on personal
computer if required by the analysis and design step
-
Hardware resources verification
-
Deployment
State of the art
Due to the interdisciplinarity of the project
it is not easy identify a unique «state of the art».
We can identify a «state of the art»
in the following fields related with the targets of the project:
-
development of a generic VHDL library
Probably industries working in hardware design have
developed their own frameworks in order to accelerate the process of design
for customers and actually many VHDL components can be bought for specific
tasks and many are free and can be downloaded from Internet but a recognised
library with an organised hierarchy of functionality is not available.
-
development of a generic VHDL framework with components
having common behaviour-related interfaces
A framework with these features has been never realised
or made available and never a standard has been created.
-
specialisation of the above framework for the development
of learning and adaptive devices
Probably theory about the concept of «smart
devices» have been explored and rules to build adaptive and learning
devices have been already written in some feasibility studies but never
a development framework (both for hardware and software) has been designed
to match these requirements.
-
research on neural networks implementation in hardware
Hardware implementations of neural networks have been
few and not successful because of the following reasons:
- missing focus on applications
- optimisation not starting from the paradigm
but only from the implementation
- poor interaction and knowledge exchange between
hardware design engineers and neural scientists
The ELAN project has the objective and all the
instruments to fill these lacks.
Potential impact
A three levels industrial exploitation is expected:
1) Industrial standard
2) VHDL library
3) FPGA based devices
End-user should be in one of the following categories:
1) small companies having know-how to design software
and hardware but having not resources to produce hardware. In the
last years, a lot of small software-houses have been able to offer
competitive solutions in a market where bigger companies where consolidated.
This fact didn’t happen for hardware because the production cost (added
anyway to design cost) is a big obstacle for small companies to be competitive
in this field. The framework should enable small companies to produce electronic
devices
-
fast time to market (no hardware production / reuse
of basic functionality from HDL library)
-
low cost (the hardware supports are commercially available
and not custom)
-
low investment risk (the hardware supports bought
are not related to a specific application)
2) big companies that have enough resources
for the production of hardware but consider a flexible and «fast
time to market» solution more adequate for devices characterised
by rapid and continuous evolution for growing performances or changing
features demand.
Generally, considering the «smart devices»
philosophy of the framework (such as devices where intelligence is embedded
and hidden at user), those companies should operate principally on automotive,
«domotics» and automation. In the first two fields the trend
is production of easy-to-use devices with a philosophy completely different
from that of personal computer.
In the past years we have seen many small software-house
companies realise products that have been competitive with products from
big companies. This situation has been possible due to the fact products
based only on software requires knowledge and not high investments in hardware
resources and instruments. So we have assisted at the birth of many small
companies in this field.
In other way we have never seen small companies
produce hardware products competitive in the market. The main problem is
that hardware production requires investments that goes over the human
resources and often some activities are realised in outsourcing. This situation
is critical for small companies and often stops their activity in the first
years of life.
The rules that enable a small company to survive
and be competitive in the market of hardware electronic products are substantially
two:
-
identify a high technology specialised market
-
realise a production framework that enables reutilization
of hardware and software components for the major number of products in
their market
The results of the project are directly tuned to realise
the two above conditions.
The market of «smart devices» intended
as devices that can work with an «hidden intelligence», is
growing in demand (automotive, domotics, monitoring) and actually not many
companies have knowledge to produce them. There is now space for a new
generation of small knowledge-based companies that could be competitive
in this field.
The role of a framework and possibly an industrial
standard would be fundamental to enable these small companies to develop
hardware devices corresponding to the features described above, at low
cost.
The major societal and economical impact should
be related with the creation of a new high technology market and a new
opportunity for entrepreneurs: this is the reason we consider an added-value
in carrying out the work at a European level.
Enabling small companies to produce intelligent
devices for domotics, automotive and automation would have a strong positive
effect in these markets. While many companies are working in industrial
automation, the situation is different for the automotive field where few
big companies have monopoly of market. The introduction of SMEs in this
market should help the evolution and accelerate the enhancement process
of electronic devices for automotive applications. The possible downgrade
of prices influenced also by a larger competition should determine an enlargement
of the application area: sophisticated devices could be applied on low
cost cars.
The domotics market is just a relative new market
and it is probably the most fertile terrain for the application of smart
devices. We have to include in this market also those devices that could
help people with handicap to interact with home environment: low cost voice
recognition devices could be embedded in any electrical appliance. Smart
devices could be a human interface for people with handicap also for communication
purpose and enable them to use efficiently personal computers and connect
to the Internet: speak to text, text to speak devices and facial expression
recognition devices could substitute keyboard, monitor and mouse. In this
way an «e-inclusion» market could be started and competition
with low cost of hardware production should be the key to have affordable
prices. Having the instruments to build these devices without hardware
production and using general purpose hardware devices (FPGA boards), a
small company could decide to specialise in this field with low investment
risk.
Contributions to standards
VHDL is a standard (VHDL-1076) developed by
the IEEE. The language has been through a couple of revisions, the most
widely used version is the 1987 (Std 1076-1987) version, sometimes referred
to as VHDL'87, but also just VHDL. However, there is a newer revision of
the language referred to as VHDL'93. VHDL'93 (adopted in 1994 of course)
is fairly new and is still in the process of replacing VHDL'87.
The project uses the VHDL standard language and
wants to produce a standard framework on top of this language. Really the
framework should have rules that enable the implementation virtually in
any kind of hardware description language but the real implementation of
the framework (the library) is realised using VHDL standard.
The main goal of the project is the creation of
an international standard for the design of smart devices that enables
reuse of components and exchangeability of hardware reducing production
costs.
Actually a generic standard that defines common
interfaces for VHDL modules related to specific functionality is missing
and normally modules implementing the same functionality have completely
different interfaces. The project is oriented to the development of smart
devices based on cognitive and neural behaviours but the framework functionality
hierarchy is really open to the addition of any new directory related to
a different field (as example telecommunication).
A non-monolithic management of the framework library
and granularity of IP should help partners of the consortium to apply the
work developed inside the project on their own typical market. In this
way the project strengthens its vocation to create a «standard»
for fast time-to-market development of smart devices instead of a simple
VHDL library.