580 California St., Suite 400
San Francisco, CA, 94104
Key research themes
1. How can neural architecture search (NAS) methods efficiently discover optimized architectures under resource or search space constraints?
This research area focuses on improving the efficiency and effectiveness of automated neural architecture search (NAS) to discover neural network models optimized not only for accuracy but also for practical deployment constraints like computational resources, latency, and search costs. It matters because NAS traditionally demands extensive computational resources and often neglects real-world constraints critical for wide adoption in edge devices and embedded systems. Recent advances explore resource-aware reward functions, search space pruning, and fast search algorithms to address these bottlenecks.
Key finding: Introduced RENA, a reinforcement learning-based NAS framework using network embedding and a policy network that progressively adapts existing models for target tasks under resource constraints. RENA devised... Read more
Key finding: Proposed Angle-Based search space Shrinking (ABS), a general, dynamic method to prune NAS search spaces by measuring angles between model weights and their initialization, which correlates better with final performance than... Read more
Key finding: Presented Improved-ENAS (I-ENAS) which enhances the Efficient NAS (ENAS) framework by modifying the reinforcement learning reward to incorporate performance relative to previously evaluated architectures. This introduces a... Read more
Key finding: Developed FPNAS, a fast NAS method that constructs full networks instead of stacking identical cells, formulating the search as a bi-level optimization solved via iterative approximation. This approach discovered... Read more
2. What are biologically inspired or brain-inspired principles that can inform neural architectures and artificial cognitive systems?
A large body of research investigates how neural architectures and cognitive models can be inspired by biological neural circuits and brain function to enhance artificial intelligence. This includes studying functional neuronal units that support perception, memory, and language, as well as cognitive architectures designed to mirror human cognition. Understanding these foundations matters for developing AI systems with improved interpretability, generalization, and potentially self-awareness—moving beyond black-box ANNs towards more explainable and human-like intelligence.
Key finding: Reviewed theoretical frameworks on how brains learn latent structured generative models to support inductive inference from sparse data, a hallmark of animal intelligence. Proposed that neural circuits implement probabilistic... Read more
Key finding: Critically examined the limitations of contemporary artificial neural networks (ANNs) by highlighting their black-box nature and lack of theoretical grounding. Advocated using simpler natural intelligence forms (e.g., in... Read more
Key finding: Provided a comprehensive overview of cognitive architectures which aim to model human cognition functionally and structurally for AI development. Distinguished between 'cognitivist' architectures focusing on modeling human... Read more
Key finding: Proposed a theory of functional neuronal units ('equimerec-units') combining threshold logic and feedback loops with backpropagation connections that sustain impulse circulation in closed circuits. These units are responsible... Read more
3. How can neural architectures and advanced learning models leverage alternative structural motifs and representations to improve learning performance and interpretability?
Exploring neural architectures beyond traditional feedforward CNNs and fully connected layers, such as tree architectures or graph-based models, can offer more efficient biological plausibility, interpretability, and performance gains. Simultaneously, alternative representations like 3D graphs permit richer modeling of complex domains like architecture design. Investigations into such structural innovations can bridge the gap between artificial models and their biological counterparts, while opening new avenues for better generalization and reduced computational overhead.
Key finding: Developed a 3-layer tree-inspired neural architecture where each weight connects to output via a single route, biologically motivated by dendritic adaptations. Demonstrated this tree architecture outperforms the 5-layer... Read more
Key finding: Introduced a graph convolutional neural network (GCNN) based technique to model designers’ aesthetic sensibility directly from 3D architectural models rather than 2D images. Created a dataset of 3D houses and columns from a... Read more
Key finding: Implemented a neuromorphic disparity estimation algorithm mimicking processing in the primate visual cortex, including Gabor filtering, binocular complex cells, divisive normalization, and center-of-mass decoding. Found that... Read more
Key finding: Detailed the classical perceptron network architecture including buffer, feature, and perceptron layers, highlighting the use of threshold neurons and supervised learning to minimize classification errors. Addressed... Read more