From crisis to opportunity

For CityU researchers Liu Yiming and Yiu Chun-ki, the cutting-edge human-machine interface (CL-HMI) system they helped to develop as part of a University co-led team during the days of the pandemic offers a fresh way to protect the health of frontline medical workers. Now they are striving to extend the system’s capabilities to widen awareness of how clinical advances can be made via the metaverse.


Based on the flexible, multi-layered electronic skin devised by the team, the Robotic Virtual Reality (VR) system facilitates the remote operation of robots. In assisting the machines to imitate even subtle motion of users, it enables robots to perform complicated tasks, such as collecting samples for COVID-19 tests and providing patient care, overcoming a shortcoming in conventional HMIs. In addition to visual feedback, sensors on the robot can provide physically felt (haptic) feedback to the user, addressing another traditional challenge.


The research is being led by Associate Professor Dr Yu Xinge, Department of Biomedical Engineering, who specialises in novel applications of flexible electronics for the biomedical field. It was the presence of Dr Yu, who joined CityU in 2018, that motivated Yiming to undertake his doctorate at the University. At the time, though, it never crossed the young researcher’s mind that he would have a chance to become involved in applications of the concepts he was to study. He now greatly appreciates his experiences, recognising the impact that applied work can have on the wider community.

While the ideas behind the Robotic VR technology were formulated before the pandemic, early thinking revolved around relatively complicated procedures. Later, it was integrated with the metaverse concept, with the ultimate goal of developing an easy-to-wear system that ensures free human movement and makes virtual-reality interconnection possible, Yiming explained. “When the team noticed that medical workers were becoming infected during the pandemic, they adjusted the research so that the new system could handle high-risk tasks, such as testing and patient care. Efficiency was also increased by making it possible for several machines to be controlled by the same person, which would enable more lives to be saved,” he said.


Chun-ki said that as life begins to normalise after the pandemic, the team is endeavouring to employ the new system to develop mixed-reality technologies to allow users to easily switch between virtual and real worlds. One goal is to reactivate senses that have been lost due to illness or other causes. “We have already developed a feedback system that could be applied in a virtual reality environment to re-train people’s sense of smell,” Chun-ki said. “We believe this could be very useful in promoting metaverse technologies and applications.”

開發人機互動系統 進軍「元宇宙」

一支城大聯合領導的科研團隊於疫情期間開發一套嶄新的硬件系統，名為閉環人機互動系統（CL-HMI），主要以團隊研發的柔性電子皮膚為基礎，每層電子皮膚之間由一系列的晶片級集成電路和傳感組件互相連結起來，傳感器可以準確感測到人體的微細動作，並將之轉化為電子訊號，經無線傳送給特定的機械人，使用者因而可以遙控機械人模仿他的動作，完成複雜任務，如為新冠病毒測試進行採樣、遙距照顧傳染病患者等。同一時間，安裝在機械人上的壓力傳感器，用於為使用者提供觸覺反饋。來自研發團隊的城大生物醫學工程系博士劉一明及博士生姚俊祺表示，研究成果有助保障前線醫護人員的健康，期望未來可擴展應用於「元宇宙」 以至與現實的無縫連接。


在城大生物醫學工程學系副教授于欣格博士的帶領下，團隊為HMI系統的靈活運用實現重大突破。劉一明指出，于教授專注研究將新型柔性電子應用在生物醫學領域，正是他感興趣的範疇，故當得悉對方於2018年加盟城大，便決定前往城大修讀博士課程，想不到更有機會參與人機互動系統的研發項目，見證其科研理念的實踐。

「關於虛實互換的研究意念在疫情前已萌生，最初的構思進行時工序較繁複，後來決定與元宇宙概念連繫起來，終極目標是開發對人類活動無阻礙、容易佩戴而又能實現虛實場景互換的新系統。」劉一明闡釋，直至疫情爆發，團隊留意到不少醫護人員受到感染，遂調整研究方向，讓新系統幫助他們進行檢測、照顧染疫病人等高危工作；同步亦能通過一人操作多台機器，提升「採樣」效率，挽救更多病人的生命安全。


疫後復常，姚俊褀指研究亦「重回正軌」，力求在擴增實境（AR）及虛擬實境（VR）以外，開發適用於「元宇宙」的混合實境（MR）技術，令使用者佩載新系統後，可隨意切換虛擬與現實。「我們目前已開發出一款用於VR的嗅覺反饋系統，其目的是在VR環境重新訓練人類的嗅覺功能，相信對應用於元宇宙的技術推進有很大幫助。」