Latest in cs.ne

total 3887took 0.12s
A Mean Field Theory of Batch NormalizationFeb 21 2019We develop a mean field theory for batch normalization in fully-connected feedforward neural networks. In so doing, we provide a precise characterization of signal propagation and gradient backpropagation in wide batch-normalized networks at initialization. ... More
A computational model for grid maps in neural populationsFeb 18 2019Feb 19 2019Grid cells in the entorhinal cortex, together with place, speed and border cells, are major contributors to the organization of spatial representations in the brain. In this contribution we introduce a novel theoretical and algorithmic framework able ... More
A computational model for grid maps in neural populationsFeb 18 2019Grid cells in the entorhinal cortex, together with place, speed and border cells, are major contributors to the organization of spatial representations in the brain. In this contribution we introduce a novel theoretical and algorithmic framework able ... More
Beyond the Memory Wall: A Case for Memory-centric HPC System for Deep LearningFeb 18 2019As the models and the datasets to train deep learning (DL) models scale, system architects are faced with new challenges, one of which is the memory capacity bottleneck, where the limited physical memory inside the accelerator device constrains the algorithm ... More
Reactive, Proactive, and Inductive Agents: An evolutionary path for biological and artificial spiking networksFeb 18 2019Complex environments provide structured yet variable sensory inputs. To best exploit information from these environments, organisms must evolve the ability to correctly anticipate consequences of unknown stimuli, and act on these predictions. We propose ... More
A One-Class Support Vector Machine Calibration Method for Time Series Change Point DetectionFeb 18 2019It is important to identify the change point of a system's health status, which usually signifies an incipient fault under development. The One-Class Support Vector Machine (OC-SVM) is a popular machine learning model for anomaly detection and hence could ... More
Differentiable reservoir computingFeb 16 2019Much effort has been devoted in the last two decades to characterize the situations in which a reservoir computing system exhibits the so called echo state and fading memory properties. These important features amount, in mathematical terms, to the existence ... More
Combination of Domain Knowledge and Deep Learning for Sentiment Analysis of Short and Informal Messages on Social MediaFeb 16 2019Sentiment analysis has been emerging recently as one of the major natural language processing (NLP) tasks in many applications. Especially, as social media channels (e.g. social networks or forums) have become significant sources for brands to observe ... More
Deep Spiking Neural Network with Spike Count based Learning RuleFeb 15 2019Deep spiking neural networks (SNNs) support asynchronous event-driven computation, massive parallelism and demonstrate great potential to improve the energy efficiency of its synchronous analog counterpart. However, insufficient attention has been paid ... More
Learning to Control Self-Assembling Morphologies: A Study of Generalization via ModularityFeb 14 2019Contemporary sensorimotor learning approaches typically start with an existing complex agent (e.g., a robotic arm), which they learn to control. In contrast, this paper investigates a modular co-evolution strategy: a collection of primitive agents learns ... More
Superposition of many models into oneFeb 14 2019We present a method for storing multiple models within a single set of parameters. Models can coexist in superposition and still be retrieved individually. In experiments with neural networks, we show that a surprisingly large number of models can be ... More
Engineered Self-Organization for Resilient Robot Self-Assembly with Minimal SurpriseFeb 14 2019In collective robotic systems, the automatic generation of controllers for complex tasks is still a challenging problem. Open-ended evolution of complex robot behaviors can be a possible solution whereby an intrinsic driver for pattern formation and self-organization ... More
Some Interesting Features of Memristor CNNFeb 14 2019In this paper, we introduce some interesting features of a memristor CNN (Cellular Neural Network). We first show that there is the similarity between the dynamics of memristors and neurons. That is, some kind of flux-controlled memristors can not respond ... More
Enhanced Robot Speech Recognition Using Biomimetic Binaural Sound Source LocalizationFeb 13 2019Inspired by the behavior of humans talking in noisy environments, we propose an embodied embedded cognition approach to improve automatic speech recognition (ASR) systems for robots in challenging environments, such as with ego noise, using binaural sound ... More
Evolutionary Algorithms for the Chance-Constrained Knapsack ProblemFeb 13 2019Evolutionary algorithms have been widely used for a range of stochastic optimization problems. In most studies, the goal is to optimize the expected quality of the solution. Motivated by real-world problems where constraint violations have extremely disruptive ... More
Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian Process Behavior, Gradient Independence, and Neural Tangent Kernel DerivationFeb 13 2019Several recent trends in machine learning theory and practice, from the design of state-of-the-art Gaussian Process to the convergence analysis of deep neural nets (DNNs) under stochastic gradient descent (SGD), have found it fruitful to study wide random ... More
A characterisation of S-box fitness landscapes in cryptographyFeb 13 2019Substitution Boxes (S-boxes) are nonlinear objects often used in the design of cryptographic algorithms. The design of high quality S-boxes is an interesting problem that attracts a lot of attention. Many attempts have been made in recent years to use ... More
Neural network models and deep learning - a primer for biologistsFeb 13 2019Originally inspired by neurobiology, deep neural network models have become a powerful tool of machine learning and artificial intelligence, where they are used to approximate functions and dynamics by learning from examples. Here we give a brief introduction ... More
Analysis of Baseline Evolutionary Algorithms for the Packing While Travelling ProblemFeb 13 2019Although the performance of base-line Evolutionary Algorithms (EAs) on linear functions has been studied rigorously, the same theoretical analyses on non-linear objectives are still far behind. In this paper, variations of the Packing While Travelling ... More
ACTRCE: Augmenting Experience via Teacher's Advice For Multi-Goal Reinforcement LearningFeb 12 2019Sparse reward is one of the most challenging problems in reinforcement learning (RL). Hindsight Experience Replay (HER) attempts to address this issue by converting a failed experience to a successful one by relabeling the goals. Despite its effectiveness, ... More
Guiding Neuroevolution with Structural ObjectivesFeb 12 2019Feb 13 2019The structure and performance of neural networks are intimately connected, and by use of evolutionary algorithms, neural network structures optimally adapted to a given task can be explored. Guiding such neuroevolution with additional objectives related ... More
Guiding Neuroevolution with Structural ObjectivesFeb 12 2019The structure and performance of neural networks are intimately connected, and by use of evolutionary algorithms, neural network structures optimally adapted to a given task can be explored. Guiding such neuroevolution with additional objectives related ... More
WiseMove: A Framework for Safe Deep Reinforcement Learning for Autonomous DrivingFeb 11 2019Machine learning can provide efficient solutions to the complex problems encountered in autonomous driving, but ensuring their safety remains a challenge. A number of authors have attempted to address this issue, but there are few publicly-available tools ... More
On Residual Networks Learning a Perturbation from IdentityFeb 11 2019The purpose of this work is to test and study the hypothesis that residual networks are learning a perturbation from identity. Residual networks are enormously important deep learning models, with many theories attempting to explain how they function; ... More
Gauge Equivariant Convolutional Networks and the Icosahedral CNNFeb 11 2019The idea of equivariance to symmetry transformations provides one of the first theoretically grounded principles for neural network architecture design. Equivariant networks have shown excellent performance and data efficiency on vision and medical imaging ... More
Interaction-Transformation Evolutionary Algorithm for Symbolic RegressionFeb 11 2019The Interaction-Transformation (IT) is a new representation for Symbolic Regression that restricts the search space into simpler, but expressive, function forms. This representation has the advantage of creating a smoother search space unlike the space ... More
Global Collaboration through Local Interaction in Competitive LearningFeb 11 2019Feature maps, that preserve the global topology of arbitrary datasets, can be formed by self-organizing competing agents. So far, it has been presumed that global interaction of agents is necessary for this process. We establish that this is not the case, ... More
Improving NeuroEvolution Efficiency by Surrogate Model-based Optimization with Phenotypic Distance KernelsFeb 09 2019In NeuroEvolution, the topologies of artificial neural networks are optimized with evolutionary algorithms to solve tasks in data regression, data classification, or reinforcement learning. One downside of NeuroEvolution is the large amount of necessary ... More
Generative Moment Matching Network-based Random Modulation Post-filter for DNN-based Singing Voice Synthesis and Neural Double-trackingFeb 09 2019This paper proposes a generative moment matching network (GMMN)-based post-filter that provides inter-utterance pitch variation for deep neural network (DNN)-based singing voice synthesis. The natural pitch variation of a human singing voice leads to ... More
Architecture CompressionFeb 08 2019In this paper we propose a novel approach to model compression termed Architecture Compression. Instead of operating on the weight or filter space of the network like classical model compression methods, our approach operates on the architecture space. ... More
A simple and efficient architecture for trainable activation functionsFeb 08 2019Feb 12 2019Learning automatically the best activation function for the task is an active topic in neural network research. At the moment, despite promising results, it is still difficult to determine a method for learning an activation function that is at the same ... More
A simple and efficient architecture for trainable activation functionsFeb 08 2019Learning automatically the best activation function for the task is an active topic in neural network research. At the moment, despite promising results, it is still difficult to determine a method for learning an activation function that is at the same ... More
Stimulating STDP to Exploit Locality for Lifelong Learning without Catastrophic ForgettingFeb 08 2019Stochastic gradient descent requires that training samples be drawn from a uniformly random distribution of the data. For a deployed system that must learn online from an uncontrolled and unknown environment, the ordering of input samples often fails ... More
Fourier Neural Networks: A Comparative StudyFeb 08 2019We review neural network architectures which were motivated by Fourier series and integrals and which are referred to as Fourier neural networks. These networks are empirically evaluated in synthetic and real-world tasks. Neither of them outperforms the ... More
Can Genetic Programming Do Manifold Learning Too?Feb 08 2019Exploratory data analysis is a fundamental aspect of knowledge discovery that aims to find the main characteristics of a dataset. Dimensionality reduction, such as manifold learning, is often used to reduce the number of features in a dataset to a manageable ... More
Understanding the One-Pixel Attack: Propagation Maps and Locality AnalysisFeb 08 2019Deep neural networks were shown to be vulnerable to single pixel modifications. However, the reason behind such phenomena has never been elucidated. Here, we propose Propagation Maps which show the influence of the perturbation in each layer of the network. ... More
Self-Adjusting Mutation Rates with Provably Optimal Success RulesFeb 07 2019The one-fifth success rule is one of the best-known and most widely accepted techniques to control the parameters of evolutionary algorithms. While it is often applied in the literal sense, a common interpretation sees the one-fifth success rule as a ... More
Toward A Neuro-inspired Creative DecoderFeb 06 2019Feb 09 2019Creativity, a process that generates novel and valuable ideas, involves increased association between task-positive (control) and task-negative (default) networks in brain. Inspired by this seminal finding, in this study we propose a creative decoder ... More
Toward A Neuro-inspired Creative DecoderFeb 06 2019Creativity, a process that generates novel and valuable ideas, involves increased association between task-positive (control) and task-negative (default) networks in brain. Inspired by this seminal finding, in this study we propose a creative decoder ... More
Investigating RNN Memory using Neuro-Evolution: Investigating Recurrent Neural Network Memory Structures using Neuro-EvolutionFeb 06 2019This paper presents a new algorithm, Evolutionary eXploration of Augmenting Memory Models (EXAMM), which is capable of evolving recurrent neural networks (RNNs) using a wide variety of memory structures, such as Delta-RNN, GRU, LSTM, MGU and UGRNN cells. ... More
Widely Linear Kernels for Complex-Valued Kernel Activation FunctionsFeb 06 2019Complex-valued neural networks (CVNNs) have been shown to be powerful nonlinear approximators when the input data can be properly modeled in the complex domain. One of the major challenges in scaling up CVNNs in practice is the design of complex activation ... More
A Generalized Framework for Population Based TrainingFeb 05 2019Population Based Training (PBT) is a recent approach that jointly optimizes neural network weights and hyperparameters which periodically copies weights of the best performers and mutates hyperparameters during training. Previous PBT implementations have ... More
How to "DODGE" Complex Software Analytics?Feb 05 2019AI software is still software. Software engineers need better tools to make better use of AI software. For example, for software defect prediction and software text mining, the default tunings for software analytics tools can be improved with "hyperparameter ... More
Deep Tree Transductions - A Short SurveyFeb 05 2019The paper surveys recent extensions of the Long-Short Term Memory networks to handle tree structures from the perspective of learning non-trivial forms of isomorph structured transductions. It provides a discussion of modern TreeLSTM models, showing the ... More
AlphaStar: An Evolutionary Computation PerspectiveFeb 05 2019In January 2019, DeepMind revealed AlphaStar to the world-the first artificial intelligence (AI) system to beat a professional player at the game of StarCraft II-representing a milestone in the progress of AI. AlphaStar draws on many areas of AI research, ... More
Total stochastic gradient algorithms and applications in reinforcement learningFeb 05 2019Backpropagation and the chain rule of derivatives have been prominent; however, the total derivative rule has not enjoyed the same amount of attention. In this work we show how the total derivative rule leads to an intuitive visual framework for creating ... More
Fatal Brain DamageFeb 05 2019The loss of a few neurons in a brain often does not result in a visible loss of function. We propose to advance the understanding of neural networks through their remarkable ability to sustain individual neuron failures, i.e. their fault tolerance. Before ... More
Online Diversity Control in Symbolic Regression via a Fast Hash-based Tree Similarity MeasureFeb 03 2019Diversity represents an important aspect of genetic programming, being directly correlated with search performance. When considered at the genotype level, diversity often requires expensive tree distance measures which have a negative impact on the algorithm's ... More
Medical Diagnosis with a Novel SVM-CoDOA Based Hybrid ApproachFeb 02 2019Machine Learning is an important sub-field of the Artificial Intelligence and it has been become a very critical task to train Machine Learning techniques via effective method or techniques. Recently, researchers try to use alternative techniques to improve ... More
Fast Re-Optimization via Structural DiversityFeb 01 2019When a problem instance is perturbed by a small modification, one would hope to find a good solution for the new instance by building on a known good solution for the previous one. Via a rigorous mathematical analysis, we show that evolutionary algorithms, ... More
Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture DesignFeb 01 2019Flow-based generative models are powerful exact likelihood models with efficient sampling and inference. Despite their computational efficiency, flow-based models generally have much worse density modeling performance compared to state-of-the-art autoregressive ... More
A New Family of Neural Networks Provably Resistant to Adversarial AttacksFeb 01 2019Adversarial attacks add perturbations to the input features with the intent of changing the classification produced by a machine learning system. Small perturbations can yield adversarial examples which are misclassified despite being virtually indistinguishable ... More
Which Surrogate Works for Empirical Performance Modelling? A Case Study with Differential EvolutionJan 30 2019It is not uncommon that meta-heuristic algorithms contain some intrinsic parameters, the optimal configuration of which is crucial for achieving their peak performance. However, evaluating the effectiveness of a configuration is expensive, as it involves ... More
The Evolved TransformerJan 30 2019Feb 15 2019Recent works have highlighted the strengths of the Transformer architecture for dealing with sequence tasks. At the same time, neural architecture search has advanced to the point where it can outperform human-designed models. The goal of this work is ... More
The Evolved TransformerJan 30 2019Feb 06 2019Recent works have highlighted the strengths of the Transformer architecture for dealing with sequence tasks. At the same time, neural architecture search has advanced to the point where it can outperform human-designed models. The goal of this work is ... More
Hardware-Guided Symbiotic Training for Compact, Accurate, yet Execution-Efficient LSTMJan 30 2019Many long short-term memory (LSTM) applications need fast yet compact models. Neural network compression approaches, such as the grow-and-prune paradigm, have proved to be promising for cutting down network complexity by skipping insignificant weights. ... More
Numerically Recovering the Critical Points of a Deep Linear AutoencoderJan 29 2019Numerically locating the critical points of non-convex surfaces is a long-standing problem central to many fields. Recently, the loss surfaces of deep neural networks have been explored to gain insight into outstanding questions in optimization, generalization, ... More
Minimax-optimal decoding of movement goals from local field potentials using complex spectral featuresJan 29 2019We consider the problem of predicting eye movement goals from local field potentials (LFP) recorded through a multielectrode array in the macaque prefrontal cortex. The monkey is tasked with performing memory-guided saccades to one of eight targets during ... More
Learning Choice FunctionsJan 29 2019We study the problem of learning choice functions, which play an important role in various domains of application, most notably in the field of economics. Formally, a choice function is a mapping from sets to sets: Given a set of choice alternatives as ... More
Heartbeat Anomaly Detection using Adversarial OversamplingJan 28 2019Cardiovascular diseases are one of the most common causes of death in the world. Prevention, knowledge of previous cases in the family, and early detection is the best strategy to reduce this fact. Different machine learning approaches to automatic diagnostic ... More
Surrogate Gradient Learning in Spiking Neural NetworksJan 28 2019A growing number of neuromorphic spiking neural network processors that emulate biological neural networks create an imminent need for methods and tools to enable them to solve real-world signal processing problems. Like conventional neural networks, ... More
Squeezed Very Deep Convolutional Neural Networks for Text ClassificationJan 28 2019Most of the research in convolutional neural networks has focused on increasing network depth to improve accuracy, resulting in a massive number of parameters which restricts the trained network to platforms with memory and processing constraints. We ... More
FPSA: A Full System Stack Solution for Reconfigurable ReRAM-based NN Accelerator ArchitectureJan 28 2019Neural Network (NN) accelerators with emerging ReRAM (resistive random access memory) technologies have been investigated as one of the promising solutions to address the \textit{memory wall} challenge, due to the unique capability of \textit{processing-in-memory} ... More
Multi Objective Particle Swarm Optimization based Cooperative Agents with Automated NegotiationJan 27 2019This paper investigates a new hybridization of multi-objective particle swarm optimization (MOPSO) and cooperative agents (MOPSO-CA) to handle the problem of stagnation encounters in MOPSO, which leads solutions to trap in local optima. The proposed approach ... More
Biologically inspired alternatives to backpropagation through time for learning in recurrent neural netsJan 25 2019The way how recurrently connected networks of spiking neurons in the brain acquire powerful information processing capabilities through learning has remained a mystery. This lack of understanding is linked to a lack of learning algorithms for recurrent ... More
Biologically inspired alternatives to backpropagation through time for learning in recurrent neural netsJan 25 2019Feb 21 2019The way how recurrently connected networks of spiking neurons in the brain acquire powerful information processing capabilities through learning has remained a mystery. This lack of understanding is linked to a lack of learning algorithms for recurrent ... More
On the Limitations of Representing Functions on SetsJan 25 2019Recent work on the representation of functions on sets has considered the use of summation in a latent space to enforce permutation invariance. In particular, it has been conjectured that the dimension of this latent space may remain fixed as the cardinality ... More
A Stable Combinatorial Particle Swarm Optimization for Scalable Feature Selection in Gene Expression DataJan 24 2019Evolutionary computation (EC) algorithms, such as discrete and multi-objective versions of particle swarm optimization (PSO), have been applied to solve the Feature selection (FS) problem, tackling the combinatorial explosion of search spaces that are ... More
Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural NetworksJan 24 2019Recent works have cast some light on the mystery of why deep nets fit any data and generalize despite being very overparametrized. This paper analyzes training and generalization for a simple 2-layer ReLU net with random initialization, and provides the ... More
Learning Vector Representation of Content and Matrix Representation of Change: Towards a Representational Model of V1Jan 24 2019This paper entertains the hypothesis that the primary purpose of the cells of the primary visual cortex (V1) is to perceive motions and predict changes of local image contents. Specifically, we propose a model that couples the vector representations of ... More
Robust computation with rhythmic spike patternsJan 23 2019Information coding by precise timing of spikes can be faster and more energy-efficient than traditional rate coding. However, spike-timing codes are often brittle, which has limited their use in theoretical neuroscience and computing applications. Here, ... More
Neural-Guided Symbolic Regression with Semantic PriorJan 23 2019Symbolic regression has been shown to be quite useful in many domains from discovering scientific laws to industrial empirical modeling. Existing methods focus on numerically fitting the given data. However, in many domains, symbolically derivable properties ... More
Perception-in-the-Loop Adversarial ExamplesJan 21 2019We present a scalable, black box, perception-in-the-loop technique to find adversarial examples for deep neural network classifiers. Black box means that our procedure only has input-output access to the classifier, and not to the internal structure, ... More
Hierarchical Attentional Hybrid Neural Networks for Document ClassificationJan 20 2019Document classification is a challenging task with important applications. Deep learning approaches to the problem have gained much attention. Despite the progress, the proposed models do not incorporate the knowledge of the document structure in the ... More
Slim LSTM networks: LSTM_6 and LSTM_C6Jan 18 2019We have shown previously that our parameter-reduced variants of Long Short-Term Memory (LSTM) Recurrent Neural Networks (RNN) are comparable in performance to the standard LSTM RNN on the MNIST dataset. In this study, we show that this is also the case ... More
Infeasibility and structural bias in Differential EvolutionJan 18 2019This paper thoroughly investigates a range of popular DE configurations to identify components responsible for the emergence of structural bias - recently identified tendency of the algorithm to prefer some regions of the search space for reasons directly ... More
Ranking Online Consumer ReviewsJan 17 2019The product reviews are posted online in the hundreds and even in the thousands for some popular products. Handling such a large volume of continuously generated online content is a challenging task for buyers, sellers, and even researchers. The purpose ... More
Evolving embodied intelligence from materials to machinesJan 17 2019Natural lifeforms specialise to their environmental niches across many levels; from low-level features such as DNA and proteins, through to higher-level artefacts including eyes, limbs, and overarching body plans. We propose Multi-Level Evolution (MLE), ... More
Interpolating Local and Global Search by Controlling the Variance of Standard Bit MutationJan 17 2019A key property underlying the success of evolutionary algorithms (EAs) is their global search behavior, which allows the algorithms to `jump' from a current state to other parts of the search space, thereby avoiding to get stuck in local optima. This ... More
Fleet Prognosis with Physics-informed Recurrent Neural NetworksJan 16 2019Services and warranties of large fleets of engineering assets is a very profitable business. The success of companies in that area is often related to predictive maintenance driven by advanced analytics. Therefore, accurate modeling, as a way to understand ... More
The Discrete Langevin Machine: Bridging the Gap Between Thermodynamic and Neuromorphic SystemsJan 16 2019A formulation of Langevin dynamics for discrete systems is derived as a new class of generic stochastic processes. The dynamics simplify for a two-state system and suggest a novel network architecture which is implemented by the Langevin machine. The ... More
Modeling neural dynamics during speech production using a state space variational autoencoderJan 13 2019Characterizing the neural encoding of behavior remains a challenging task in many research areas due in part to complex and noisy spatiotemporal dynamics of evoked brain activity. An important aspect of modeling these neural encodings involves separation ... More
Low-Power Neuromorphic Hardware for Signal Processing ApplicationsJan 11 2019Machine learning has emerged as the dominant tool for implementing complex cognitive tasks that require supervised, unsupervised, and reinforcement learning. While the resulting machines have demonstrated in some cases even super-human performance, their ... More
Large-scale Collaborative Filtering with Product EmbeddingsJan 11 2019The application of machine learning techniques to large-scale personalized recommendation problems is a challenging task. Such systems must make sense of enormous amounts of implicit feedback in order to understand user preferences across numerous product ... More
Using fuzzy bits and neural networks to partially invert few rounds of some cryptographic hash functionsJan 08 2019We consider fuzzy, or continuous, bits, which take values in [0;1] and (-1;1] instead of {0;1}, and operations on them (NOT, XOR etc.) and on their sequences (ADD), to obtain the generalization of cryptographic hash functions, CHFs, for the messages consisting ... More
Solar-Sail Trajectory Design of Multiple Near Earth Asteroids Exploration Based on Deep Neural NetworkJan 08 2019Jan 16 2019In the preliminary trajectory design of the multi-target rendezvous problem, a model that can quickly estimate the cost of the orbital transfer is essential. The estimation of the transfer time using solar sail between two arbitrary orbits is difficult ... More
Learning the optimal state-feedback via supervised imitation learningJan 07 2019Imitation learning is a control design paradigm that seeks to learn a control policy reproducing demonstrations from experts. By substituting expert's demonstrations for optimal behaviours, the same paradigm leads to the design of control policies closely ... More
Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their SolutionsJan 07 2019Jan 09 2019While the history of machine learning so far encompasses a series of problems posed by researchers and algorithms that learn their solutions, an important question is whether the problems themselves can be generated by the algorithm at the same time as ... More
Multi-Objective Reinforced Evolution in Mobile Neural Architecture SearchJan 04 2019Jan 16 2019Fabricating neural models for a wide range of mobile devices demands for a specific design of networks due to highly constrained resources. Both evolution algorithms (EA) and reinforced learning methods (RL) have been dedicated to solve neural architecture ... More
A Unified Theory of Early Visual Representations from Retina to Cortex through Anatomically Constrained Deep CNNsJan 03 2019The visual system is hierarchically organized to process visual information in successive stages. Neural representations vary drastically across the first stages of visual processing: at the output of the retina, ganglion cell receptive fields (RFs) exhibit ... More
Self-supervised Learning of Image Embedding for Continuous ControlJan 03 2019Operating directly from raw high dimensional sensory inputs like images is still a challenge for robotic control. Recently, Reinforcement Learning methods have been proposed to solve specific tasks end-to-end, from pixels to torques. However, these approaches ... More
Subspace Match Probably Does Not Accurately Assess the Similarity of Learned RepresentationsJan 03 2019Learning informative representations of data is one of the primary goals of deep learning, but there is still little understanding as to what representations a neural network actually learns. To better understand this, subspace match was recently proposed ... More
Learning Nonlinear State Space Models with Hamiltonian Sequential Monte Carlo SamplerJan 03 2019State space models (SSM) have been widely applied for the analysis and visualization of large sequential datasets. Sequential Monte Carlo (SMC) is a very popular particle-based method to sample latent states from intractable posteriors. However, SSM is ... More
Generating Multiple Objects at Spatially Distinct LocationsJan 03 2019Recent improvements to Generative Adversarial Networks (GANs) have made it possible to generate realistic images in high resolution based on natural language descriptions such as image captions. Furthermore, conditional GANs allow us to control the image ... More
A Constrained Cooperative Coevolution Strategy for Weights Adaptation Optimization of Heterogeneous Epidemic Spreading NetworksJan 03 2019In this paper, the dynamic constrained optimization problem of weights adaptation for heterogeneous epidemic spreading networks is investigated. Due to the powerful ability of searching global optimum, evolutionary algorithms are employed as the optimizers. ... More
Performance of Three Slim Variants of The Long Short-Term Memory (LSTM) LayerJan 02 2019The Long Short-Term Memory (LSTM) layer is an important advancement in the field of neural networks and machine learning, allowing for effective training and impressive inference performance. LSTM-based neural networks have been successfully employed ... More
General Subpopulation Framework and Taming the Conflict Inside PopulationsJan 02 2019Structured evolutionary algorithms have been investigated for some time. However, they have been under-explored specially in the field of multi-objective optimization. Despite their good results, the use of complex dynamics and structures make their understanding ... More
Opportunistic Learning: Budgeted Cost-Sensitive Learning from Data StreamsJan 02 2019Feb 18 2019In many real-world learning scenarios, features are only acquirable at a cost constrained under a budget. In this paper, we propose a novel approach for cost-sensitive feature acquisition at the prediction-time. The suggested method acquires features ... More
Transfer learning from language models to image caption generators: Better models may not transfer betterJan 01 2019When designing a neural caption generator, a convolutional neural network can be used to extract image features. Is it possible to also use a neural language model to extract sentence prefix features? We answer this question by trying different ways to ... More
Mid-Level Visual Representations Improve Generalization and Sample Efficiency for Learning Active TasksDec 31 2018One of the ultimate promises of computer vision is to help robotic agents perform active tasks, like delivering packages or doing household chores. However, the conventional approach to solving "vision" is to define a set of offline recognition problems ... More