Use this URL to cite or link to this record in EThOS: https://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.780179
Title: Parallel circuit : a modular neural network architecture
Author: Phan, Kien Tuong
ISNI:       0000 0004 7965 8680
Awarding Body: University of Nottingham
Current Institution: University of Nottingham
Date of Award: 2019
Availability of Full Text:
Access from EThOS:
Full text unavailable from EThOS. Restricted access.
Access from Institution:
Abstract:
One of the obstacles that hinder the development of Artificial Neural Networks (ANNs) is the heavy computational cost of the training process. In an attempt to address this problem, I proposed a lightweight model named Parallel Circuits (PCs), with an emphasis on modularity. One of the key inspirations for the proposed model is the human retina, which consists of various cell types that only respond to particular visual stimuli. Similarly, conventional ANNs with high redundancy are decomposed into semi-independent modules, which is deemed to provide more efficient learning, both in terms of speed and generalizability. Owing to the benefits of having fewer connections, the PC models were empirically shown to be considerably faster, especially when implemented in larger models. I also pursued the ability of automatic problem decomposition, and discovered that diversifying the learning process in each circuit strongly benefits the generalization of the proposed model. PC was shown to be advantageous in term of sparsity, which is highly correlated to modularity. DropCircuit, a regularizer that targets circuits, was introduced to further enhance their specialities. Together with PCs, DropCircuit outperformed models with dense connectivity in several experiments. The circuit-level DropCircuit also exhibited better performance compared to conventional DropOut in conjunction with both PC and non-PC configurations, demonstrating the benefits of modularity. The modularity was further enhanced by imposing a set of biologically inspired constraints. Circuits are modelled as either excitatory or inhibitory types with contrastive properties. Modified PC networks were shown to discover sparse and part-based representations, showing further improvement in generalization.
Supervisor: Not available Sponsor: Not available
Qualification Name: Thesis (Ph.D.) Qualification Level: Doctoral
EThOS ID: uk.bl.ethos.780179  DOI: Not available
Keywords: QA 75 Electronic computers. Computer science
Share: