Latest in cs.ne

total 6123took 0.12s
Decentralised Multi-Demic Evolutionary Approach to the Dynamic Multi-Agent Travelling Salesman ProblemJun 13 2019The Travelling Salesman and its variations are some of the most well known NP hard optimisation problems. This paper looks to use both centralised and decentralised implementations of Evolutionary Algorithms (EA) to solve a dynamic variant of the Multi-Agent ... More
Meta-heuristic for non-homogeneous peak density spaces and implementation on 2 real-world parameter learning/tuning applicationsJun 13 2019Observer effect in physics (/psychology) regards bias in measurement (/perception) due to the interference of instrument (/knowledge). Based on these concepts, a new meta-heuristic algorithm is proposed for controlling memory usage per localities without ... More
Neural Graph Evolution: Towards Efficient Automatic Robot DesignJun 12 2019Despite the recent successes in robotic locomotion control, the design of robot relies heavily on human engineering. Automatic robot design has been a long studied subject, but the recent progress has been slowed due to the large combinatorial search ... More
MOPED: Efficient priors for scalable variational inference in Bayesian deep neural networksJun 12 2019Variational inference for Bayesian deep neural networks (DNNs) requires specifying priors and approximate posterior distributions for neural network weights. Specifying meaningful weight priors is a challenging problem, particularly for scaling variational ... More
Learning Curves for Deep Neural Networks: A Gaussian Field Theory PerspectiveJun 12 2019A series of recent works suggest that deep neural networks (DNNs), of fixed depth, are equivalent to certain Gaussian Processes (NNGP/NTK) in the highly over-parameterized regime (width or number-of-channels going to infinity). Other works suggest that ... More
Task Agnostic Continual Learning via Meta LearningJun 12 2019While neural networks are powerful function approximators, they suffer from catastrophic forgetting when the data distribution is not stationary. One particular formalism that studies learning under non-stationary distribution is provided by continual ... More
Run-Time Efficient RNN Compression for Inference on Edge DevicesJun 12 2019Recurrent neural networks can be large and compute-intensive, yet many applications that benefit from RNNs run on small devices with very limited compute and storage capabilities while still having run-time constraints. As a result, there is a need for ... More
Medium-Term Load Forecasting Using Support Vector Regression, Feature Selection, and Symbiotic Organism Search OptimizationJun 11 2019An accurate load forecasting has always been one of the main indispensable parts in the operation and planning of power systems. Among different time horizons of forecasting, while short-term load forecasting (STLF) and long-term load forecasting (LTLF) ... More
StRE: Self Attentive Edit Quality Prediction in WikipediaJun 11 2019Wikipedia can easily be justified as a behemoth, considering the sheer volume of content that is added or removed every minute to its several projects. This creates an immense scope, in the field of natural language processing towards developing automated ... More
Principled Training of Neural Networks with Direct Feedback AlignmentJun 11 2019The backpropagation algorithm has long been the canonical training method for neural networks. Modern paradigms are implicitly optimized for it, and numerous guidelines exist to ensure its proper use. Recently, synthetic gradients methods -where the error ... More
Unsupervised Minimax: Adversarial Curiosity, Generative Adversarial Networks, and Predictability MinimizationJun 11 2019Generative Adversarial Networks (GANs) learn to model data distributions through two unsupervised neural networks, each minimizing the objective function maximized by the other. We relate this game theoretic strategy to earlier neural networks playing ... More
Classification of EEG Signals using Genetic Programming for Feature ConstructionJun 11 2019The analysis of electroencephalogram (EEG) waves is of critical importance for the diagnosis of sleep disorders, such as sleep apnea and insomnia, besides that, seizures, epilepsy, head injuries, dizziness, headaches and brain tumors. In this context, ... More
Subspace Attack: Exploiting Promising Subspaces for Query-Efficient Black-box AttacksJun 11 2019Unlike the white-box counterparts that are widely studied and readily accessible, adversarial examples in black-box settings are generally more Herculean on account of the difficulty of estimating gradients. Many methods achieve the task by issuing numerous ... More
Weight Agnostic Neural NetworksJun 11 2019Not all neural network architectures are created equal, some perform much better than others for certain tasks. But how important are the weight parameters of a neural network compared to its architecture? In this work, we question to what extent neural ... More
BowNet: Dilated Convolution Neural Network for Ultrasound Tongue Contour ExtractionJun 10 2019Ultrasound imaging is safe, relatively affordable, and capable of real-time performance. One application of this technology is to visualize and to characterize human tongue shape and motion during a real-time speech to study healthy or impaired speech ... More
Data-driven Reconstruction of Nonlinear Dynamics from Sparse ObservationJun 10 2019We present a data-driven model to reconstruct nonlinear dynamics from a very sparse times series data, which relies on the strength of the echo state network (ESN) in learning nonlinear representation of data. With an assumption of the universal function ... More
The Riddle of TogelbyJun 10 2019At the 2017 Artificial and Computational Intelligence in Games meeting at Dagstuhl, Julian Togelius asked how to make spaces where every way of filling in the details yielded a good game. This study examines the possibility of enriching search spaces ... More
E-LPIPS: Robust Perceptual Image Similarity via Random Transformation EnsemblesJun 10 2019Jun 11 2019It has been recently shown that the hidden variables of convolutional neural networks make for an efficient perceptual similarity metric that accurately predicts human judgment on relative image similarity assessment. First, we show that such learned ... More
Autonomous Goal Exploration using Learned Goal Spaces for Visuomotor Skill Acquisition in RobotsJun 10 2019The automatic and efficient discovery of skills, without supervision, for long-living autonomous agents, remains a challenge of Artificial Intelligence. Intrinsically Motivated Goal Exploration Processes give learning agents a human-inspired mechanism ... More
Exploration and Exploitation in Symbolic Regression using Quality-Diversity and Evolutionary Strategies AlgorithmsJun 10 2019By combining Genetic Programming, MAP-Elites and Covariance Matrix Adaptation Evolution Strategy, we demonstrate very high success rates in Symbolic Regression problems. MAP-Elites is used to improve exploration while preserving diversity and avoiding ... More
Differentiable Surface Splatting for Point-based Geometry ProcessingJun 10 2019We propose Differentiable Surface Splatting (DSS), a high-fidelity differentiable renderer for point clouds. Gradients for point locations and normals are carefully designed to handle discontinuities of the rendering function. Regularization terms are ... More
Convolutional Bipartite Attractor NetworksJun 08 2019In human perception and cognition, the fundamental operation that brains perform is interpretation: constructing coherent neural states from noisy, incomplete, and intrinsically ambiguous evidence. The problem of interpretation is well matched to an early ... More
Class-specific Differential Detection in Diffractive Optical Neural Networks Improves Inference AccuracyJun 08 2019Diffractive deep neural networks have been introduced earlier as an optical machine learning framework that uses task-specific diffractive surfaces designed by deep learning to all-optically perform inference, achieving promising performance for object ... More
Non-Differentiable Supervised Learning with Evolution Strategies and Hybrid MethodsJun 07 2019In this work we show that Evolution Strategies (ES) are a viable method for learning non-differentiable parameters of large supervised models. ES are black-box optimization algorithms that estimate distributions of model parameters; however they have ... More
AutoGrow: Automatic Layer Growing in Deep Convolutional NetworksJun 07 2019We propose AutoGrow to automate depth discovery in Deep Neural Networks (DNNs): starting from a shallow seed architecture, AutoGrow grows new layers if the growth improves the accuracy; otherwise, the growth stops and the network depth is discovered. ... More
Stochasticity and Robustness in Spiking Neural NetworksJun 06 2019Artificial neural networks normally require precise weights to operate, despite their origins in biological systems, which can be highly variable and noisy. When implementing artificial networks which utilize analog 'synaptic' devices to encode weights, ... More
One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizersJun 06 2019The success of lottery ticket initializations (Frankle and Carbin, 2019) suggests that small, sparsified networks can be trained so long as the network is initialized appropriately. Unfortunately, finding these "winning ticket" initializations is computationally ... More
Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLPJun 06 2019The lottery ticket hypothesis proposes that over-parameterization of deep neural networks (DNNs) aids training by increasing the probability of a "lucky" sub-network initialization being present rather than by helping the optimization process. This phenomenon ... More
Training large-scale ANNs on simulated resistive crossbar arraysJun 06 2019Accelerating training of artificial neural networks (ANN) with analog resistive crossbar arrays is a promising idea. While the concept has been verified on very small ANNs and toy data sets (such as MNIST), more realistically sized ANNs and datasets have ... More
A method for the classification of chimera states of coupled oscillators and its application for creating a neural network information converterJun 06 2019The paper presents a new method for the classification of chimera states, which characterizes the synchronization of two coupled oscillators more accurately. As an example of method application, a neural network information converter based on a network ... More
Mutual Information and the Edge of Chaos in Reservoir ComputersJun 06 2019A reservoir computer is a dynamical system that may be used to perform computations. A reservoir computer usually consists of a set of nonlinear nodes coupled together in a network so that there are feedback paths. Training the reservoir computer consists ... More
Localizing Catastrophic Forgetting in Neural NetworksJun 06 2019Artificial neural networks (ANNs) suffer from catastrophic forgetting when trained on a sequence of tasks. While this phenomenon was studied in the past, there is only very limited recent research on this phenomenon. We propose a method for determining ... More
Non-uniqueness phenomenon of object representation in modelling IT cortex by deep convolutional neural network (DCNN)Jun 06 2019Recently DCNN (Deep Convolutional Neural Network) has been advocated as a general and promising modelling approach for neural object representation in primate inferotemporal cortex. In this work, we show that some inherent non-uniqueness problem exists ... More
Introducing languid particle dynamics to a selection of PSO variantsJun 06 2019Previous research showed that conditioning a PSO agent's movement based on its personal fitness improvement enhances the standard PSO method. In this article, languid particle dynamics (LPD) technique is used on five adequate and widely used PSO variants. ... More
Evolution of Hierarchical Structure & Reuse in iGEM Synthetic DNA SequencesJun 06 2019Many complex systems, both in technology and nature, exhibit hierarchical modularity: smaller modules, each of them providing a certain function, are used within larger modules that perform more complex functions. Previously, we have proposed a modeling ... More
Deep Reinforcement Learning for Multi-objective OptimizationJun 06 2019This study proposes an end-to-end framework for solving multi-objective optimization problems (MOPs) using Deep Reinforcement Learning (DRL), termed DRL-MOA. The idea of decomposition is adopted to decompose a MOP into a set of scalar optimization subproblems. ... More
On the use of Pairwise Distance Learning for Brain Signal Classification with Limited ObservationsJun 05 2019The increasing access to brain signal data using electroencephalography creates new opportunities to study electrophysiological brain activity and perform ambulatory diagnoses of neuronal diseases. This work proposes a pairwise distance learning approach ... More
Genetic Random Weight Change Algorithm for the Learning of Multilayer Neural NetworksJun 05 2019A new method to improve the performance of Random weight change (RWC) algorithm based on a simple genetic algorithm, namely, Genetic random weight change (GRWC) is proposed. It is to find the optimal values of global minima via learning. In contrast to ... More
The Unreasonable Effectiveness of Transformer Language Models in Grammatical Error CorrectionJun 04 2019Recent work on Grammatical Error Correction (GEC) has highlighted the importance of language modeling in that it is certainly possible to achieve good performance by comparing the probabilities of the proposed edits. At the same time, advancements in ... More
Neuromorphic Architecture Optimization for Task-Specific Dynamic LearningJun 04 2019The ability to learn and adapt in real time is a central feature of biological systems. Neuromorphic architectures demonstrating such versatility can greatly enhance our ability to efficiently process information at the edge. A key challenge, however, ... More
Finding Syntactic Representations in Neural StacksJun 04 2019Neural network architectures have been augmented with differentiable stacks in order to introduce a bias toward learning hierarchy-sensitive regularities. It has, however, proven difficult to assess the degree to which such a bias is effective, as the ... More
Hamiltonian Neural NetworksJun 04 2019Even though neural networks enjoy widespread use, they still struggle to learn the basic laws of physics. How might we endow them with better inductive biases? In this paper, we draw inspiration from Hamiltonian mechanics to train models that learn and ... More
Disentangling neural mechanisms for perceptual groupingJun 04 2019Forming perceptual groups and individuating objects in visual scenes is an essential step towards visual intelligence. This ability is thought to arise in the brain from computations implemented by bottom-up, horizontal, and top-down connections between ... More
PCA-driven Hybrid network design for enabling Intelligence at the EdgeJun 04 2019The recent advent of IOT has increased the demand for enabling AI-based edge computing in several applications including healthcare monitoring systems, autonomous vehicles etc. This has necessitated the search for efficient implementations of neural networks ... More
Options as responses: Grounding behavioural hierarchies in multi-agent RLJun 04 2019We propose a novel hierarchical agent architecture for multi-agent reinforcement learning with concealed information. The hierarchy is grounded in the concealed information about other players, which resolves "the chicken or the egg" nature of option ... More
Options as responses: Grounding behavioural hierarchies in multi-agent RLJun 04 2019Jun 06 2019We propose a novel hierarchical agent architecture for multi-agent reinforcement learning with concealed information. The hierarchy is grounded in the concealed information about other players, which resolves "the chicken or the egg" nature of option ... More
Kinetic Market Model: An Evolutionary AlgorithmJun 04 2019This research proposes the econophysics kinetic market model as an evolutionary algorithm's instance. The immediate results from this proposal is a new replacement rule for family competition genetic algorithms. It also represents a starting point to ... More
Do place cells dream of conditional probabilities? Learning Neural Nyström representationsJun 03 2019We posit that hippocampal place cells encode information about future locations under a transition distribution observed as an agent explores a given (physical or conceptual) space. The encoding of information about the current location, usually associated ... More
A detailed study of recurrent neural networks used to model tasks in the cerebral cortexJun 03 2019We studied the properties of simple recurrent neural networks trained to perform temporal tasks and also flow control tasks with temporal stimulus. We studied mainly three aspects: inner configuration sets, memory capacity with the scale of the models ... More
Neural networks grown and self-organized by noiseJun 03 2019Living neural networks emerge through a process of growth and self-organization that begins with a single cell and results in a brain, an organized and functional computational device. Artificial neural networks, however, rely on human-designed, hand-programmed ... More
A Perspective on Objects and Systematic Generalization in Model-Based RLJun 03 2019In order to meet the diverse challenges in solving many real-world problems, an intelligent agent has to be able to dynamically construct a model of its environment. Objects facilitate the modular reuse of prior knowledge and the combinatorial construction ... More
Learning Perceptually-Aligned Representations via Adversarial RobustnessJun 03 2019Many applications of machine learning require models that are human-aligned, i.e., that make decisions based on human-meaningful information about the input. We identify the pervasive brittleness of deep networks' learned representations as a fundamental ... More
Learning to solve the credit assignment problemJun 03 2019Backpropagation is driving today's artificial neural networks (ANNs). However, despite extensive research, it remains unclear if the brain implements this algorithm. Among neuroscientists, reinforcement learning (RL) algorithms are often seen as a realistic ... More
SpikeGrad: An ANN-equivalent Computation Model for Implementing Backpropagation with SpikesJun 03 2019Event-based neuromorphic systems promise to reduce the energy consumption of deep learning tasks by replacing expensive floating point operations on dense matrices by low power sparse and asynchronous operations on spike events. While these systems can ... More
Discovering Neural WiringsJun 03 2019The success of neural networks has driven a shift in focus from feature engineering to architecture engineering. However, successful networks today are constructed using a small and manually defined set of building blocks. Even in methods of neural architecture ... More
Push and Pull Search Embedded in an M2M Framework for Solving Constrained Multi-objective Optimization ProblemsJun 02 2019In dealing with constrained multi-objective optimization problems (CMOPs), a key issue of multi-objective evolutionary algorithms (MOEAs) is to balance the convergence and diversity of working populations.
Neural Bug Finding: A Study of Opportunities and ChallengesJun 01 2019Static analysis is one of the most widely adopted techniques to find software bugs before code is put in production. Designing and implementing effective and efficient static analyses is difficult and requires high expertise, which results in only a few ... More
Improved memory in recurrent neural networks with sequential non-normal dynamicsMay 31 2019Training recurrent neural networks (RNNs) is a hard problem due to degeneracies in the optimization landscape, a problem also known as the vanishing/exploding gradients problem. Short of designing new RNN architectures, various methods for dealing with ... More
Implicit Regularization in Deep Matrix FactorizationMay 31 2019Efforts to understand the generalization mystery in deep learning have led to the belief that gradient-based optimization induces a form of implicit regularization, a bias towards models of low "complexity." We study the implicit regularization of gradient ... More
Implicit Regularization in Deep Matrix FactorizationMay 31 2019Jun 04 2019Efforts to understand the generalization mystery in deep learning have led to the belief that gradient-based optimization induces a form of implicit regularization, a bias towards models of low "complexity." We study the implicit regularization of gradient ... More
Updates of Equilibrium Prop Match Gradients of Backprop Through Time in an RNN with Static InputMay 31 2019Equilibrium Propagation (EP) is a biologically inspired learning algorithm for convergent recurrent neural networks, i.e. RNNs that are fed by a static input x and settle to a steady state. Training convergent RNNs consists in adjusting the weights until ... More
Interval timing in deep reinforcement learning agentsMay 31 2019The measurement of time is central to intelligent behavior. We know that both animals and artificial agents can successfully use temporal dependencies to select actions. In artificial agents, little work has directly addressed (1) which architectural ... More
DeepShift: Towards Multiplication-Less Neural NetworksMay 30 2019Deep learning models, especially DCNN have obtained high accuracies in several computer vision applications. However, for deployment in mobile environments, the high computation and power budget proves to be a major bottleneck. Convolution layers and ... More
DeepShift: Towards Multiplication-Less Neural NetworksMay 30 2019Jun 06 2019Deep learning models, especially DCNN have obtained high accuracies in several computer vision applications. However, for deployment in mobile environments, the high computation and power budget proves to be a major bottleneck. Convolution layers and ... More
Epsilon-Lexicase Selection for RegressionMay 30 2019Lexicase selection is a parent selection method that considers test cases separately, rather than in aggregate, when performing parent selection. It performs well in discrete error spaces but not on the continuous-valued problems that compose most system ... More
What Can Neural Networks Reason About?May 30 2019May 31 2019Neural networks have successfully been applied to solving reasoning tasks, ranging from learning simple concepts like "close to", to intricate questions whose reasoning procedures resemble algorithms. Empirically, not all network structures work equally ... More
AssembleNet: Searching for Multi-Stream Neural Connectivity in Video ArchitecturesMay 30 2019Learning to represent videos is a very challenging task both algorithmically and computationally. Standard video CNN architectures have been designed by directly extending architectures devised for image understanding to a third dimension (using a limited ... More
Factorized Inference in Deep Markov Models for Incomplete Multimodal Time SeriesMay 30 2019Integrating deep learning with latent state space models has the potential to yield temporal models that are powerful, yet tractable and interpretable. Unfortunately, current models are not designed to handle missing data or multiple data modalities, ... More
A Hippocampus Model for Online One-Shot Storage of Pattern SequencesMay 30 2019We present a computational model based on the CRISP theory (Content Representation, Intrinsic Sequences, and Pattern completion) of the hippocampus that allows to continuously store pattern sequences online in a one-shot fashion. Rather than storing a ... More
Quantifying the alignment of graph and features in deep learningMay 30 2019We show that the classification performance of Graph Convolutional Networks is related to the alignment between features, graph and ground truth, which we quantify using a subspace alignment measure corresponding to the Frobenius norm of the matrix of ... More
AlignFlow: Cycle Consistent Learning from Multiple Domains via Normalizing FlowsMay 30 2019Given unpaired data from multiple domains, a key challenge is to efficiently exploit these data sources for modeling a target domain. Variants of this problem have been studied in many contexts, such as cross-domain translation and domain adaptation. ... More
Bandlimiting Neural Networks Against Adversarial AttacksMay 30 2019In this paper, we study the adversarial attack and defence problem in deep learning from the perspective of Fourier analysis. We first explicitly compute the Fourier transform of deep ReLU neural networks and show that there exist decaying but non-zero ... More
Nonvolatile Spintronic Memory Cells for Neural NetworksMay 29 2019A new spintronic nonvolatile memory cell analogous to 1T DRAM with non-destructive read is proposed. The cells can be used as neural computing units. A dual-circuit neural network architecture is proposed to leverage these devices against the complex ... More
Size-free generalization bounds for convolutional neural networksMay 29 2019We prove bounds on the generalization error of convolutional networks. The bounds are in terms of the training loss, the number of parameters, the Lipschitz constant of the loss and the distance from the weights to the initial weights. They are independent ... More
Size-free generalization bounds for convolutional neural networksMay 29 2019Jun 12 2019We prove bounds on the generalization error of convolutional networks. The bounds are in terms of the training loss, the number of parameters, the Lipschitz constant of the loss and the distance from the weights to the initial weights. They are independent ... More
Are Disentangled Representations Helpful for Abstract Visual Reasoning?May 29 2019A disentangled representation encodes information about the salient factors of variation in the data independently. Although it is often argued that this representational format is useful in learning to solve many real-world up-stream tasks, there is ... More
Attention Based Pruning for Shift NetworksMay 29 2019In many application domains such as computer vision, Convolutional Layers (CLs) are key to the accuracy of deep learning methods. However, it is often required to assemble a large number of CLs, each containing thousands of parameters, in order to reach ... More
On the Expressive Power of Deep Polynomial Neural NetworksMay 29 2019We study deep neural networks with polynomial activations, particularly their expressive power. For a fixed architecture and activation degree, a polynomial neural network defines an algebraic map from weights to polynomials. The image of this map is ... More
Composing Neural Algorithms with FuguMay 28 2019Neuromorphic hardware architectures represent a growing family of potential post-Moore's Law Era platforms. Largely due to event-driving processing inspired by the human brain, these computer platforms can offer significant energy benefits compared to ... More
Harnessing Slow Dynamics in Neuromorphic ComputationMay 28 2019Neuromorphic Computing is a nascent research field in which models and devices are designed to process information by emulating biological neural systems. Thanks to their superior energy efficiency, analog neuromorphic systems are highly promising for ... More
Unified Probabilistic Deep Continual Learning through Generative Replay and Open Set RecognitionMay 28 2019We introduce a unified probabilistic approach for deep continual learning based on variational Bayesian inference with open set recognition. Our model combines a probabilistic encoder with a generative model and a generative linear classifier that get ... More
Supervised Learning in Spiking Neural Networks with Phase-Change Memory SynapsesMay 28 2019Spiking neural networks (SNN) are artificial computational models that have been inspired by the brain's ability to naturally encode and process information in the time domain. The added temporal dimension is believed to render them more computationally ... More
Network DeconvolutionMay 28 2019Convolution is a central operation in Convolutional Neural Networks (CNNs), which applies a kernel or mask to overlapping regions shifted across the image. In this work we show that the underlying kernels are trained with highly correlated data, which ... More
Autonomous skill discovery with Quality-Diversity and Unsupervised DescriptorsMay 28 2019Quality-Diversity optimization is a new family of optimization algorithms that, instead of searching for a single optimal solution to solving a task, searches for a large collection of solutions that all solve the task in a different way. This approach ... More
Overlearning Reveals Sensitive AttributesMay 28 2019`Overlearning' means that a model trained for a seemingly simple objective implicitly learns to recognize attributes that are (1) statistically uncorrelated with the objective, and (2) sensitive from a privacy or bias perspective. For example, a binary ... More
SGD on Neural Networks Learns Functions of Increasing ComplexityMay 28 2019We perform an experimental study of the dynamics of Stochastic Gradient Descent (SGD) in learning deep neural networks for several real and synthetic classification tasks. We show that in the initial epochs, almost all of the performance improvement of ... More
Inference with Hybrid Bio-hardware Neural NetworksMay 28 2019To understand the learning process in brains, biologically plausible algorithms have been explored by modeling the detailed neuron properties and dynamics. On the other hand, simplified multi-layer models of neural networks have shown great success on ... More
Efficient Network Construction through Structural PlasticityMay 27 2019Deep Neural Networks (DNNs) on hardware is facing excessive computation cost due to the massive number of parameters. A typical training pipeline to mitigate over-parameterization is to pre-define a DNN structure first with redundant learning units (filters ... More
Improved Training Speed, Accuracy, and Data Utilization Through Loss Function OptimizationMay 27 2019As the complexity of neural network models has grown, it has become increasingly important to optimize their design automatically through metalearning. Methods for discovering hyperparameters, topologies, and learning rate schedules have lead to significant ... More
Understanding Generalization of Deep Neural Networks Trained with Noisy LabelsMay 27 2019Over-parameterized deep neural networks trained by simple first-order methods are known to be able to fit any labeling of data. When the training dataset contains a fraction of noisy labels, can neural networks be resistant to over-fitting and still generalize ... More
Understanding Generalization of Deep Neural Networks Trained with Noisy LabelsMay 27 2019May 29 2019Over-parameterized deep neural networks trained by simple first-order methods are known to be able to fit any labeling of data. Such over-fitting ability hinders generalization when mislabeled training examples are present. On the other hand, simple regularization ... More
CIF: Continuous Integrate-and-Fire for End-to-End Speech RecognitionMay 27 2019Automatic speech recognition (ASR) system is undergoing an exciting pathway to be more simplified and practical with the spring up of various end-to-end models. However, the mainstream of them neglects the positioning of token boundaries from continuous ... More
Incremental Learning Using a Grow-and-Prune Paradigm with Efficient Neural NetworksMay 27 2019Deep neural networks (DNNs) have become a widely deployed model for numerous machine learning applications. However, their fixed architecture, substantial training cost, and significant model redundancy make it difficult to efficiently update them to ... More
Seeing Convolution Through the Eyes of Finite Transformation Semigroup Theory: An Abstract Algebraic Interpretation of Convolutional Neural NetworksMay 26 2019Researchers are actively trying to gain better insights into the representational properties of convolutional neural networks for guiding better network designs and for interpreting a network's computational nature. Gaining such insights can be an arduous ... More
A hybrid model for predicting human physical activity status from lifelogging dataMay 26 2019One trend in the recent healthcare transformations is people are encouraged to monitor and manage their health based on their daily diets and physical activity habits. However, much attention of the use of operational research and analytical models in ... More
A Staged Approach to Evolving Real-world UAV ControllersMay 26 2019A testbed has recently been introduced that evolves controllers for arbitrary hover-capable UAVs, with evaluations occurring directly on the robot. To prepare the testbed for real-world deployment, we investigate the effects of state-space limitations ... More
ProbAct: A Probabilistic Activation Function for Deep Neural NetworksMay 26 2019Activation functions play an important role in the training of artificial neural networks and the Rectified Linear Unit (ReLU) has been the mainstream in recent years. Most of the activation functions currently used are deterministic in nature, whose ... More
Efficient Neural Task Adaptation by Maximum Entropy InitializationMay 25 2019Transferring knowledge from one neural network to another has been shown to be helpful for learning tasks with few training examples. Prevailing fine-tuning methods could potentially contaminate pre-trained features by comparably high energy random noise. ... More
Lifelong Neural Predictive Coding: Sparsity Yields Less Forgetting when Learning CumulativelyMay 25 2019In lifelong learning systems, especially those based on artificial neural networks, one of the biggest obstacles is the severe inability to retain old knowledge as new information is encountered. This phenomenon is known as catastrophic forgetting. In ... More
A neuromorphic boost to RNNs using low pass filtersMay 25 2019The increasing difficulty with Moore's law scaling and the remarkable success of machine learning have triggered a renaissance in the study of low-latency, energy-efficient accelerators for machine learning applications. In particular, spiking neural ... More