Neural Simulator
Abstract
I. Introduction
Almost all human actions come from brain signals. As the brain have functions uniform with the computer's central processing unit (CPU), it supervises all the organs which corresponds to I/O (input/output) devices in computer systems. Occasionally the brain, acting as a control center for complicated human system, may be deprived their links between brain and spinal cord due to unfortunate incident of various nervous system diseases such as disease names. According to published statistics, the spinal cord injury patients were reported approximately 250,000 cases in the U. S. and 11,000 cases of new patient is occurred in each year. Most patients are young, on average 28.7 years old. 52.4 % of them are complete paralysis. The cost of care for high tetraplegia is $741, 425 in the 1st year, $132, 807 in subsequent year (source: NSCISC). Another examples. If there is a severe condition, these patients cannot move their whole body anymore, and it is called as a locked in syndrome.
Even if the brain lose it's ability to control human organs because of these variety of reasons, reported data in many cases testifies that the quality of brain signals are still remains as normal people does. Thus, brain machine interface may present proper solutions for those patients who cannot move their body as their own desire in spite of unmarred brain signals. Various recording methods, such as EEG and ECoG can be applied for measuring brain signals, and those derived signals can substitute for those from abnormal I/O systems of human.
Potential clinical applications using the BMI technology range from the control of the computer, a robotic actuator, a wheel chair to environment control such as a TV set, a CD player and so on. Over the last decade, a number of non-human and human studies have driven the rapid development of BMIs [5-10]. For example, one could control a computer cursor in the one dimensional or the two dimensional workspace [5, 7, 9-10]. A recent study showed that a monkey used the brain signal to grasp food with a robotic arm [8]. A different study also showed that monkeys could control their paralyzed hand by controlling arm muscles using the brain [11]. It has been also demonstrated that human subjects drove a wheel chair [12] or typewrote texts using a speller [13-14] only with their thought. For decades, various studies have made the rapid development of the brain machine interface.
However, despite these advances, brain machine interface has some limitations. For example, most BMI studies are based on the offline study, and it may not apply to real life. Because these offline studies used to collect data at once, the subjects cannot check their performance at all. This disadvantage lowers user's possibility of modify their strategies to compensate for their control. In real life, physical activities are done by real-time communication between the brain and other organs, these offline studies are contrary to the natural laws. For real life applicable BMIs, online and closed-loop studies are essential factors to reduce practical errors and disparities.
In online BMI studies, brain signals measured from each subjects are reflected to system in real-time and the subjects can receive immediate feedback about their manuplation results. It has the advantage that the subjects can modify their strategies and actions in real time.These BMI studies suggested that online BMI studies may be used in rehabilitation equipment and devices such as robotic arms or wheelchairs. However, despite of possibilities of online studies, it still remains a few problems to solve.
The current closed-loop BMI experiment still accompanies with a number of practical issues, including high cost, time limitation, subject training and system setup and maintenance. For instance, it would typically take several hours to setup, calibrate, and run a BMI system with a subject to investigate a single question, assuming that the subject has been trained to use the BMI before. If an experiment requires a repetitive set of trials with the change of parameters and models, it would take multiple days of experimental sessions, increasing the cost of experiment drastically
In order to solve these practical issues, this study proposes a new concept of software, the neural simulator. The simulator uses mouse movement, instead of real brain signals. Created data which is made by mouse movement is transformed into neural activity like form and it is used in the virtual environment.
II. Method
The neural simulator can be tested in a simple environment. In this experiment, the subjects were asked to control wireless mouse followed by guidance on the dual monitor. Both online and offline studies focus on hit the green target which randomly turns up on the monitor. Since this neural simulator aimed at rehabilitation of tetraplagic patients, the neural activities converted by mouse control should reflect natural tendency of BMI targeted patients. Thus, the cursor movements are hidden while neural parameter generation or training phase are on going. Under these circumstances, various investments can be conducted for clarifying parameters diversifying effects or feedback effect.
A. Simulation procedures
Neural cursor control with the BMI simulator consists of three phases: 1) open-loop training phase; 2) closed-loop training phase; and 3) closed-loop testing phase. When we operate the simulator, we can choose various type of models - for example, we can change the kinematic variable as position or velocity to generate the neural activity, choose the decoding algorithm (e.g. Kalman filter or Linear filter), and finally choose whether we select the closed-loop training phase or not. The simulation begins with the open-loop (OL) training phase. Two canvases appear separately on the single or dual monitor OL training. The first canvas is used to hold a user‟s mouse input in which the cursor controlled by the mouse is invisible. In this way, the user does not receive visual feedback of how he/she moves a mouse. This mimics a condition of real BMI users where they cannot observe their movements while they intend some action. The second canvas is used to guide the user to move the mouse following the movement of a training cursor. The motion of the training cursor is programmed based on the bell-shaped velocity profile. The training cursor travels over a straight line between two distant points randomly placed on the screen. Once the training cursor reaches a target point, it starts to move to the next target point. The user is shown the training cursor, the starting point and the target point, each marked either by circle or cross. The user is instructed to move a mouse to follow the training cursor movement. Through this process, the BMI simulator generates the synthetic firing data using the kinematics of the mouse movement on the first canvas, synchronizes it with the training cursor kinematics, and builds a decoder with these synchronized data sets. The closed-loop (CL) training phase starts after building a decoder (Fig 2B-C). Similar to the OL training phase, a user's mouse movement is held on the first canvas without visibility. The second canvas, however, shows not only the training cursor but the feedback cursor which the user controls him/herself. This feedback cursor provides the user with the feedback of how well he/she follows the training cursor with the decoder built from the previous training phase. By viewing the difference between the training cursor and the feedback cursor, the user may adjust mouse control to reduce the error. At the end of every CL training trial, the simulator shows the error measured in terms of the mean distance between the training and the feedback cursors. The BMI simulator also updates the decoder‟s parameter values using the most recent training samples obtained from the last CL training trial. After all training phases are finished, the simulator runs through the closed-loop (CL) testing phase (Fig 2D). In this phase, only one cursor controlled by the user is shown on the second canvas along with a randomly located target. The user‟s task is to move the cursor using the trained decoder and mouse control to reach the target within time limit. A trial ends either when the user hits the target or when the movement time exceeds time limit. Next trial begins with a new random target. The cursor starts from the previous target position regardless of the success of target acquisition. This trial is repeated multiple times until a sufficient number of cursor control data are collected.
Neural activity generation
Various behavior features are under neural firing..
시뮬레이션 순서
뉴럴 시뮬레이터는 기본적으로 총 3단계로 구성되어 있으며: 1) 오픈 루프 트레이닝 단계, 2) 클로즈드 루프 트레이닝 단계, 그리고 3) 클로즈드 루프 테스팅 단계 이다. 사용자의 연습 환경에 따른 효과를 테스트 하기 위해 4) 프리 플레이 단계가 추가되었다. 본 뉴럴 시뮬레이터를 가동함으로써, 우리는 리얼 익스피리먼트 전에 다양한 모델들을 저비용으로 테스트 해 볼 수 있다. 예를 들면 뉴럴 액티비티의 생성을 속도로 할 것인지, 위치로 할 것인지, 디코딩 파라메터는 어떤 것으로 결정할 것인지, 그리고 클로즈드 루프 트레이닝을 포함할 것인지, 그리고 보상을 줄 것인지 말 것인지에 대한.
시뮬레이터는 항상 오픈루프 트레이닝과 함께 시작한다.
III.
A. Hardware condition
B. Software condition
i. BMI Simulator intro
ii. Neural activity generation
iii. Simulation procedure
C. Subjects
i.
IV. Result
V. Discussion
