Latest in cs.ne

total 3290took 0.27s
A Generalization Method of Partitioned Activation Function for Complex NumberFeb 08 2018A method to convert real number partitioned activation function into complex number one is provided. The method has 4em variations; 1 has potential to get holomorphic activation, 2 has potential to conserve complex angle, and the last 1 guarantees interaction ... More
Using a reservoir computer to learn chaotic attractors, with applications to chaos synchronisation and cryptographyFeb 08 2018Using the machine learning approach known as reservoir computing, it is possible to train one dynamical system to emulate another. We show that such trained reservoir computers reproduce the properties of the attractor of the chaotic system sufficiently ... More
Biological Mechanisms for Learning: A Computational Model of Olfactory Learning in the Manduca sexta Moth, with Applications to Neural NetsFeb 08 2018The insect olfactory system, which includes the antennal lobe (AL), mushroom body (MB), and ancillary structures, is a relatively simple neural system capable of learning. Its structural features, which are widespread in biological neural systems, process ... More
Classification of Things in DBpedia using Deep Neural NetworksFeb 07 2018The Semantic Web aims at representing knowledge about the real world at web scale - things, their attributes and relationships among them can be represented as nodes and edges in an inter-linked semantic graph. In the presence of noisy data, as is typical ... More
Energy-Efficient CMOS Memristive Synapses for Mixed-Signal Neuromorphic System-on-a-ChipFeb 07 2018Emerging non-volatile memory (NVM), or memristive, devices promise energy-efficient realization of deep learning, when efficiently integrated with mixed-signal integrated circuits on a CMOS substrate. Even though several algorithmic challenges need to ... More
Universal Deep Neural Network CompressionFeb 07 2018Compression of deep neural networks (DNNs) for memory- and computation-efficient compact feature representations becomes a critical problem particularly for deployment of DNNs on resource-limited platforms. In this paper, we investigate lossy compression ... More
Granger-causal Attentive Mixtures of ExpertsFeb 06 2018Several methods have recently been proposed to detect salient input features for outputs of neural networks. Those methods offer a qualitative glimpse at feature importance, but they fall short of providing quantifiable attributions that can be compared ... More
Brain-inspired photonic signal processor for periodic pattern generation and chaotic system emulationFeb 06 2018Reservoir computing is a bio-inspired computing paradigm for processing time-dependent signals. Its hardware implementations have received much attention because of their simplicity and remarkable performance on a series of benchmark tasks. In previous ... More
Regularized Evolution for Image Classifier Architecture SearchFeb 05 2018Feb 06 2018The effort devoted to hand-crafting image classifiers has motivated the use of architecture search to discover them automatically. Reinforcement learning and evolution have both shown promise for this purpose. This study employs a regularized version ... More
Deep Predictive Models in Interactive MusicJan 31 2018Automatic music generation is a compelling task where much recent progress has been made with deep learning models. In this paper, we ask how these models can be integrated into interactive music systems; how can they encourage or enhance the music making ... More
Stochastic Downsampling for Cost-Adjustable Inference and Improved Regularization in Convolutional NetworksJan 29 2018It is desirable to train convolutional networks (CNNs) to run more efficiently during inference. In many cases however, the computational budget that the system has for inference cannot be known beforehand during training, or the inference budget is dependent ... More
Psychlab: A Psychology Laboratory for Deep Reinforcement Learning AgentsJan 24 2018Feb 04 2018Psychlab is a simulated psychology laboratory inside the first-person 3D game world of DeepMind Lab (Beattie et al. 2016). Psychlab enables implementations of classical laboratory psychological experiments so that they work with both human and artificial ... More
Improving TSP Solutions Using GA with a New Hybrid Mutation Based on Knowledge and RandomnessJan 22 2018Genetic algorithm (GA) is an efficient tool for solving optimization problems by evolving solutions, as it mimics the Darwinian theory of natural evolution. The mutation operator is one of the key success factors in GA, as it is considered the exploration ... More
A Deep Reinforcement Learning Chatbot (Short Version)Jan 20 2018We present MILABOT: a deep reinforcement learning chatbot developed by the Montreal Institute for Learning Algorithms (MILA) for the Amazon Alexa Prize competition. MILABOT is capable of conversing with humans on popular small talk topics through both ... More
Better Runtime Guarantees Via Stochastic DominationJan 13 2018Apart from few exceptions, the mathematical runtime analysis of evolutionary algorithms is mostly concerned with expected runtimes. In this work, we argue that stochastic domination is a notion that should be used more frequently in this area. Stochastic ... More
An Incremental Self-Organizing Architecture for Sensorimotor Learning and PredictionDec 22 2017During visuomotor tasks, robots have to compensate for the temporal delays inherent in their sensorimotor processing systems. This capability becomes crucial in a dynamic environment where the visual input is constantly changing, e.g. when interacting ... More
Detection and classification of masses in mammographic images in a multi-kernel approachDec 20 2017According to the World Health Organization, breast cancer is the main cause of cancer death among adult women in the world. Although breast cancer occurs indiscriminately in countries with several degrees of social and economic development, among developing ... More
Analysis of supervised and semi-supervised GrowCut applied to segmentation of masses in mammography imagesDec 20 2017Breast cancer is already one of the most common form of cancer worldwide. Mammography image analysis is still the most effective diagnostic method to promote the early detection of breast cancer. Accurately segmenting tumors in digital mammography images ... More
"Zero-Shot" Super-Resolution using Deep Internal LearningDec 17 2017Deep Learning has led to a dramatic leap in Super-Resolution (SR) performance in the past few years. However, being supervised, these SR methods are restricted to specific training data, where the acquisition of the low-resolution (LR) images from their ... More
Adaptation to criticality through organizational invariance in embodied agentsDec 13 2017Many biological and cognitive systems do not operate deep within one or other regime of activity. Instead, they are poised at critical points located at transitions of their parameter space. The pervasiveness of criticality suggests that there may be ... More
CNNs are Globally Optimal Given Multi-Layer SupportDec 07 2017Dec 14 2017Stochastic Gradient Descent (SGD) is the central workhorse for training modern CNNs. Although giving impressive empirical performance it can be slow to converge. In this paper we explore a novel strategy for training a CNN using an alternation strategy ... More
Dialectical Multispectral Classification of Diffusion-Weighted Magnetic Resonance Images as an Alternative to Apparent Diffusion Coefficients Maps to Perform Anatomical AnalysisDec 03 2017Multispectral image analysis is a relatively promising field of research with applications in several areas, such as medical imaging and satellite monitoring. A considerable number of current methods of analysis are based on parametric statistics. Alternatively, ... More
Triagem virtual de imagens de imuno-histoquímica usando redes neurais artificiais e espectro de padrõesDec 03 2017The importance of organizing medical images according to their nature, application and relevance is increasing. Furhermore, a previous selection of medical images can be useful to accelerate the task of analysis by pathologists. Herein this work we propose ... More
Fuzzy-Based Dialectical Non-Supervised Image Classification and ClusteringDec 03 2017The materialist dialectical method is a philosophical investigative method to analyze aspects of reality. These aspects are viewed as complex processes composed by basic units named poles, which interact with each other. Dialectics has experienced considerable ... More
A semi-supervised fuzzy GrowCut algorithm to segment and classify regions of interest of mammographic imagesDec 03 2017According to the World Health Organization, breast cancer is the most common form of cancer in women. It is the second leading cause of death among women round the world, becoming the most fatal form of cancer. Mammographic image segmentation is a fundamental ... More
Reconstruction of Electrical Impedance Tomography Using Fish School Search, Non-Blind Search, and Genetic AlgorithmDec 03 2017Electrical Impedance Tomography (EIT) is a noninvasive imaging technique that does not use ionizing radiation, with application both in environmental sciences and in health. Image reconstruction is performed by solving an inverse problem and ill-posed. ... More
Evaluation of Alzheimer's Disease by Analysis of MR Images using Multilayer Perceptrons and Kohonen SOM Classifiers as an Alternative to the ADC MapsDec 03 2017Alzheimer's disease is the most common cause of dementia, yet hard to diagnose precisely without invasive techniques, particularly at the onset of the disease. This work approaches image analysis and classification of synthetic multispectral images composed ... More
Adversarial Networks for Prostate Cancer DetectionNov 28 2017The large number of trainable parameters of deep neural networks renders them inherently data hungry. This characteristic heavily challenges the medical imaging community and to make things even worse, many imaging modalities are ambiguous in nature leading ... More
Population Based Training of Neural NetworksNov 27 2017Nov 28 2017Neural networks dominate the modern machine learning landscape, but their training and success still suffer from sensitivity to empirical choices of hyperparameters such as model architecture, loss function, and optimisation algorithm. In this work we ... More
Neural Networks Architecture Evaluation in a Quantum ComputerNov 13 2017In this work, we propose a quantum algorithm to evaluate neural networks architectures named Quantum Neural Network Architecture Evaluation (QNNAE). The proposed algorithm is based on a quantum associative memory and the learning algorithm for artificial ... More
Searching for Biophysically Realistic Parameters for Dynamic Neuron Models by Genetic Algorithms from Calcium Imaging RecordingNov 04 2017Individual Neurons in the nervous systems exploit various dynamics. To capture these dynamics for single neurons, we tune the parameters of an electrophysiological model of nerve cells, to fit experimental data obtained by calcium imaging. A search for ... More
Convolutional Drift Networks for Video ClassificationNov 03 2017Analyzing spatio-temporal data like video is a challenging task that requires processing visual and temporal information effectively. Convolutional Neural Networks have shown promise as baseline fixed feature extractors through transfer learning, a technique ... More
PDE-Net: Learning PDEs from DataOct 26 2017Jan 01 2018In this paper, we present an initial attempt to learn evolution PDEs from data. Inspired by the latest development of neural network designs in deep learning, we propose a new feed-forward deep network, called PDE-Net, to fulfill two objectives at the ... More
Progressive Learning for Systematic Design of Large Neural NetworksOct 23 2017We develop an algorithm for systematic design of a large artificial neural network using a progression property. We find that some non-linear functions, such as the rectifier linear unit and its derivatives, hold the property. The systematic design addresses ... More
Model based learning for accelerated, limited-view 3D photoacoustic tomographyAug 31 2017Recent advances in deep learning for tomographic reconstructions have shown great potential to create accurate and high quality images with a considerable speed-up. In this work we present a deep neural network that is specifically designed to provide ... More
Efficient Noisy Optimisation with the Sliding Window Compact Genetic AlgorithmAug 07 2017The compact genetic algorithm is an Estimation of Distribution Algorithm for binary optimisation problems. Unlike the standard Genetic Algorithm, no cross-over or mutation is involved. Instead, the compact Genetic Algorithm uses a virtual population represented ... More
Deepest Neural NetworksJul 09 2017This paper shows that a long chain of perceptrons (that is, a multilayer perceptron, or MLP, with many hidden layers of width one) can be a universal classifier. The classification procedure is not necessarily computationally efficient, but the technique ... More
Fatiguing STDP: Learning from Spike-Timing Codes in the Presence of Rate CodesJun 17 2017Spiking neural networks (SNNs) could play a key role in unsupervised machine learning applications, by virtue of strengths related to learning from the fine temporal structure of event-based signals. However, some spike-timing-related strengths of SNNs ... More
Self-adaptive node-based PCA encodingsJun 16 2017In this paper we propose an algorithm, Simple Hebbian PCA, and prove that it is able to calculate the principal component analysis (PCA) in a distributed fashion across nodes. It simplifies existing network structures by removing intralayer weights, essentially ... More
Evaluating Noisy Optimisation Algorithms: First Hitting Time is ProblematicJun 13 2017Jul 12 2017A key part of any evolutionary algorithm is fitness evaluation. When fitness evaluations are corrupted by noise, as happens in many real-world problems as a consequence of various types of uncertainty, a strategy is needed in order to cope with this. ... More
Free energy-based reinforcement learning using a quantum processorMay 29 2017Recent theoretical and experimental results suggest the possibility of using current and near-future quantum hardware in challenging sampling tasks. In this paper, we introduce free energy-based reinforcement learning (FERL) as an application of quantum ... More
Fast-Slow Recurrent Neural NetworksMay 24 2017Jun 09 2017Processing sequential data of variable length is a major challenge in a wide range of applications, such as speech recognition, language modeling, generative image modeling and machine translation. Here, we address this challenge by proposing a novel ... More
Forced to Learn: Discovering Disentangled Representations Without Exhaustive LabelsMay 01 2017Learning a better representation with neural networks is a challenging problem, which was tackled extensively from different prospectives in the past few years. In this work, we focus on learning a representation that could be used for a clustering task ... More
Explaining How a Deep Neural Network Trained with End-to-End Learning Steers a CarApr 25 2017As part of a complete software stack for autonomous driving, NVIDIA has created a neural-network-based system, known as PilotNet, which outputs steering angles given images of the road ahead. PilotNet is trained using road images paired with the steering ... More
Population Seeding Techniques for Rolling Horizon Evolution in General Video Game PlayingApr 23 2017While Monte Carlo Tree Search and closely related methods have dominated General Video Game Playing, recent research has demonstrated the promise of Rolling Horizon Evolutionary Algorithms as an interesting alternative. However, there is little attention ... More
The True Destination of EGO is Multi-local OptimizationApr 19 2017Efficient global optimization is a popular algorithm for the optimization of expensive multimodal black-box functions. One important reason for its popularity is its theoretical foundation of global convergence. However, as the budgets in expensive optimization ... More
Criticality as It Could Be: organizational invariance as self-organized criticality in embodied agentsApr 18 2017May 24 2017This paper outlines a methodological approach for designing adaptive agents driving themselves near points of criticality. Using a synthetic approach we construct a conceptual model that, instead of specifying mechanistic requirements to generate criticality, ... More
In-Datacenter Performance Analysis of a Tensor Processing UnitApr 16 2017Many architects believe that major improvements in cost-energy-performance must now come from domain-specific hardware. This paper evaluates a custom ASIC---called a Tensor Processing Unit (TPU)---deployed in datacenters since 2015 that accelerates the ... More
Multi-Period Flexibility Forecast for Low Voltage ProsumersMar 26 2017Nov 08 2017Near-future electric distribution grids operation will have to rely on demand-side flexibility, both by implementation of demand response strategies and by taking advantage of the intelligent management of increasingly common small-scale energy storage. ... More
Pattern representation and recognition with accelerated analog neuromorphic systemsMar 17 2017Jul 03 2017Despite being originally inspired by the central nervous system, artificial neural networks have diverged from their biological archetypes as they have been remodeled to fit particular tasks. In this paper, we review several possibilites to reverse map ... More
Tree Memory Networks for Modelling Long-term Temporal DependenciesMar 12 2017In the domain of sequence modelling, Recurrent Neural Networks (RNN) have been capable of achieving impressive results in a variety of application areas including visual question answering, part-of-speech tagging and machine translation. However this ... More
Robustness from structure: Inference with hierarchical spiking networks on analog neuromorphic hardwareMar 12 2017How spiking networks are able to perform probabilistic inference is an intriguing question, not only for understanding information processing in the brain, but also for transferring these computational principles to neuromorphic silicon circuits. A number ... More
Integer Factorization with a Neuromorphic SieveMar 10 2017The bound to factor large integers is dominated by the computational effort to discover numbers that are smooth, typically performed by sieving a polynomial sequence. On a von Neumann architecture, sieving has log-log amortized time complexity to check ... More
Understanding Synthetic Gradients and Decoupled Neural InterfacesMar 01 2017When training neural networks, the use of Synthetic Gradients (SG) allows layers or modules to be trained without update locking - without waiting for a true error gradient to be backpropagated - resulting in Decoupled Neural Interfaces (DNIs). This unlocked ... More
Soft + Hardwired Attention: An LSTM Framework for Human Trajectory Prediction and Abnormal Event DetectionFeb 18 2017As humans we possess an intuitive ability for navigation which we master through years of practice; however existing approaches to model this trait for diverse tasks including monitoring pedestrian flow and detecting abnormal events have been limited ... More
Estimation of classrooms occupancy using a multi-layer perceptronFeb 07 2017This paper presents a multi-layer perceptron model for the estimation of classrooms number of occupants from sensed indoor environmental data-relative humidity, air temperature, and carbon dioxide concentration. The modelling datasets were collected from ... More
Learning Criticality in an Embodied Boltzmann MachineFeb 02 2017Many biological and cognitive systems do not operate deep into one or other regime of activity. Instead, they exploit critical surfaces poised at transitions in their parameter space. The pervasiveness of criticality in natural systems suggests that there ... More
Reinforcement Learning Using Quantum Boltzmann MachinesDec 17 2016Dec 25 2016We investigate whether quantum annealers with select chip layouts can outperform classical computers in reinforcement learning tasks. We associate a transverse field Ising spin Hamiltonian with a layout of qubits similar to that of a deep Boltzmann machine ... More
Coupling Distributed and Symbolic Execution for Natural Language QueriesDec 08 2016Building neural networks to query a knowledge base (a table) with natural language is an emerging research topic in NLP. The neural enquirer typically necessitates multiple steps of execution because of the compositionality of queries. In previous studies, ... More
Learning in the Machine: Random Backpropagation and the Learning ChannelDec 08 2016Random backpropagation (RBP) is a variant of the backpropagation algorithm for training neural networks, where the transpose of the forward matrices are replaced by fixed random matrices in the calculation of the weight updates. It is remarkable both ... More
Towards better decoding and language model integration in sequence to sequence modelsDec 08 2016The recently proposed Sequence-to-Sequence (seq2seq) framework advocates replacing complex data processing pipelines, such as an entire automatic speech recognition system, with a single neural network trained in an end-to-end fashion. In this contribution, ... More
Geometric Decomposition of Feed Forward Neural NetworksDec 08 2016There have been several attempts to mathematically understand neural networks and many more from biological and computational perspectives. The field has exploded in the last decade, yet neural networks are still treated much like a black box. In this ... More
Improving the Performance of Neural Machine Translation Involving Morphologically Rich LanguagesDec 07 2016The advent of the attention mechanism in neural machine translation models has improved the performance of machine translation systems by enabling selective lookup into the source sentence. In this paper, the efficiencies of translation using bidirectional ... More
Neural Turing Machines: Convergence of Copy TasksDec 07 2016The architecture of neural Turing machines is differentiable end to end and is trainable with gradient descent methods. Due to their large unfolded depth Neural Turing Machines are hard to train and because of their linear access of complete memory they ... More
A simple and efficient SNN and its performance & robustness evaluation method to enable hardware implementationDec 07 2016Spiking Neural Networks (SNN) are more closely related to brain-like computation and inspire hardware implementation. This is enabled by small networks that give high performance on standard classification problems. In literature, typical SNNs are deep ... More
Mode Regularized Generative Adversarial NetworksDec 07 2016Although Generative Adversarial Networks achieve state-of-the-art results on a variety of generative tasks, they are regarded as highly unstable and prone to miss modes. We argue that these bad behaviors of GANs are due to the very particular functional ... More
Semi-Supervised Learning with the Deep Rendering Mixture ModelDec 06 2016Semi-supervised learning algorithms reduce the high cost of acquiring labeled training data by using both labeled and unlabeled data during learning. Deep Convolutional Networks (DCNs) have achieved great success in supervised tasks and as such have been ... More
Correlation Alignment for Unsupervised Domain AdaptationDec 06 2016In this chapter, we present CORrelation ALignment (CORAL), a simple yet effective method for unsupervised domain adaptation. CORAL minimizes domain shift by aligning the second-order statistics of source and target distributions, without requiring any ... More
A Probabilistic Framework for Deep LearningDec 06 2016We develop a probabilistic framework for deep learning based on the Deep Rendering Mixture Model (DRMM), a new generative probabilistic model that explicitly capture variations in data due to latent task nuisance variables. We demonstrate that max-sum ... More
Statistical mechanics of unsupervised feature learning in a restricted Boltzmann machine with binary synapsesDec 06 2016Revealing hidden features in unlabeled data is called unsupervised feature learning, which plays an important role in pretraining a deep neural network. Here we provide a statistical mechanics analysis of the unsupervised learning in a restricted Boltzmann ... More
Improving the Performance of Neural Networks in Regression Tasks Using DraweringDec 05 2016The method presented extends a given regression neural network to make its performance improve. The modification affects the learning procedure only, hence the extension may be easily omitted during evaluation without any change in prediction. It means ... More
Towards the Limit of Network QuantizationDec 05 2016Network quantization is one of network compression techniques employed to reduce the redundancy of deep neural networks. It compresses the size of the storage for a large number of network parameters in a neural network by quantizing them and encoding ... More
BrainFrame: A heterogeneous accelerator platform for neuron simulationsDec 05 2016Objective: The advent of High-Performance Computing (HPC) in recent years has led to its increasing use in brain study through computational models. The scale and complexity of such models are constantly increasing, leading to challenging computational ... More
BrainFrame: A heterogeneous accelerator platform for neuron simulationsDec 05 2016Dec 06 2016Objective: The advent of High-Performance Computing (HPC) in recent years has led to its increasing use in brain study through computational models. The scale and complexity of such models are constantly increasing, leading to challenging computational ... More
Enabling Bio-Plausible Multi-level STDP using CMOS Neurons with Dendrites and Bistable RRAMsDec 05 2016Large-scale integration of emerging nanoscale non-volatile memory devices, e.g. resistive random-access memory (RRAM), can enable a new generation of neuromorphic computers that can solve a wide range of machine learning problems. Such hybrid CMOS-RRAM ... More
Message Passing Multi-Agent GANsDec 05 2016Communicating and sharing intelligence among agents is an important facet of achieving Artificial General Intelligence. As a first step towards this challenge, we introduce a novel framework for image generation: Message Passing Multi-Agent Generative ... More
Known Unknowns: Uncertainty Quality in Bayesian Neural NetworksDec 05 2016Dec 23 2016We evaluate the uncertainty quality in neural networks using anomaly detection. We extract uncertainty measures (e.g. entropy) from the predictions of candidate models, use those measures as features for an anomaly detector, and gauge how well the detector ... More
Known Unknowns: Uncertainty Quality in Bayesian Neural NetworksDec 05 2016We evaluate the uncertainty quality in neural networks using anomaly detection. We extract uncertainty measures (e.g. entropy) from the predictions of candidate models, use those measures as features for an anomaly detector, and gauge how well the detector ... More
Semi-supervised learning of deep metrics for stereo reconstructionDec 03 2016Deep-learning metrics have recently demonstrated extremely good performance to match image patches for stereo reconstruction. However, training such metrics requires large amount of labeled stereo images, which can be difficult or costly to collect for ... More
Positive blood culture detection in time series data using a BiLSTM networkDec 03 2016The presence of bacteria or fungi in the bloodstream of patients is abnormal and can lead to life-threatening conditions. A computational model based on a bidirectional long short-term memory artificial neural network, is explored to assist doctors in ... More
Parameter Compression of Recurrent Neural Networks and Degredation of Short-term MemoryDec 02 2016The significant computational costs of deploying neural networks in large-scale or resource constrained environments, such as data centers and mobile devices, has spurred interest in model compression, which can achieve a reduction in both arithmetic ... More
Summary - TerpreT: A Probabilistic Programming Language for Program InductionDec 02 2016We study machine learning formulations of inductive program synthesis; that is, given input-output examples, synthesize source code that maps inputs to corresponding outputs. Our key contribution is TerpreT, a domain-specific language for expressing program ... More
Cognitive Deep Machine Can Train ItselfDec 02 2016Machine learning is making substantial progress in diverse applications. The success is mostly due to advances in deep learning. However, deep learning can make mistakes and its generalization abilities to new tasks are questionable. We ask when and how ... More
Probabilistic Neural ProgramsDec 02 2016We present probabilistic neural programs, a framework for program induction that permits flexible specification of both a computational model and inference algorithm while simultaneously enabling the use of deep neural networks. Probabilistic neural programs ... More
Unsupervised learning of image motion by recomposing sequencesDec 01 2016We propose a new method for learning a representation of image motion in an unsupervised fashion. We do so by learning an image sequence embedding that respects associativity and invertibility properties of composed sequences with known temporal order. ... More
Multi-modal Variational Encoder-DecodersDec 01 2016Recent advances in neural variational inference have facilitated efficient training of powerful directed graphical models with continuous latent variables, such as variational autoencoders. However, these models usually assume simple, uni-modal priors ... More
Adversarial Images for Variational AutoencodersDec 01 2016We investigate adversarial attacks for autoencoders. We propose a procedure that distorts the input image to mislead the autoencoder in reconstructing a completely different target image. We attack the internal latent representations, attempting to make ... More
Reliable Evaluation of Neural Network for Multiclass Classification of Real-world DataNov 30 2016This paper presents a systematic evaluation of Neural Network (NN) for classification of real-world data. In the field of machine learning, it is often seen that a single parameter that is 'predictive accuracy' is being used for evaluating the performance ... More
Capacity and Trainability in Recurrent Neural NetworksNov 29 2016Two potential bottlenecks on the expressiveness of recurrent neural networks (RNNs) are their ability to store information about the task in their parameters, and to store information about the input history in their units. We show experimentally that ... More
Multi-objective Active Control Policy Design for Commensurate and Incommensurate Fractional Order Chaotic Financial SystemsNov 29 2016In this paper, an active control policy design for a fractional order (FO) financial system is attempted, considering multiple conflicting objectives. An active control template as a nonlinear state feedback mechanism is developed and the controller gains ... More
Fractional Order Load-Frequency Control of Interconnected Power Systems Using Chaotic Multi-objective OptimizationNov 29 2016Fractional order proportional-integral-derivative (FOPID) controllers are designed for load frequency control (LFC) of two interconnected power systems. Conflicting time domain design objectives are considered in a multi objective optimization (MOO) based ... More
Fractional Order AGC for Distributed Energy Resources Using Robust OptimizationNov 29 2016The applicability of fractional order (FO) automatic generation control (AGC) for power system frequency oscillation damping is investigated in this paper, employing distributed energy generation. The hybrid power system employs various autonomous generation ... More
Intelligible Language Modeling with Input Switched Affine NetworksNov 28 2016The computational mechanisms by which nonlinear recurrent neural networks (RNNs) achieve their goals remains an open question. There exist many problem domains where intelligibility of the network model is crucial for deployment. Here we introduce a recurrent ... More
Emergence of foveal image sampling from learning to attend in visual scenesNov 28 2016We describe a neural attention model with a learnable retinal sampling lattice. The model is trained on a visual search task requiring the classification of an object embedded in a visual scene amidst background distractors using the smallest number of ... More
Dense Prediction on Sequences with Time-Dilated Convolutions for Speech RecognitionNov 28 2016In computer vision pixelwise dense prediction is the task of predicting a label for each pixel in the image. Convolutional neural networks achieve good performance on this task, while being computationally efficient. In this paper we carry these ideas ... More
Efficient Convolutional Auto-Encoding via Random Convexification and Frequency-Domain MinimizationNov 28 2016The omnipresence of deep learning architectures such as deep convolutional neural networks (CNN)s is fueled by the synergistic combination of ever-increasing labeled datasets and specialized hardware. Despite the indisputable success, the reliance on ... More
Dynamic landscape models of coevolutionary gamesNov 28 2016Players of coevolutionary games may update not only their strategies but also their networks of interaction. Based on interpreting the payoff of players as fitness, dynamic landscape models are proposed. The modeling procedure is carried out for Prisoner's ... More
Kernel classification of connectomes based on earth mover's distance between graph spectraNov 27 2016In this paper, we tackle a problem of predicting phenotypes from structural connectomes. We propose that normalized Laplacian spectra can capture structural properties of brain networks, and hence graph spectral distributions are useful for a task of ... More
Learning Python Code Suggestion with a Sparse Pointer NetworkNov 24 2016To enhance developer productivity, all modern integrated development environments (IDEs) include code suggestion functionality that proposes likely next tokens at the cursor. While current IDEs work well for statically-typed languages, their reliance ... More
Survey of Expressivity in Deep Neural NetworksNov 24 2016We survey results on neural network expressivity described in "On the Expressive Power of Deep Neural Networks". The paper motivates and develops three natural measures of expressiveness, which all display an exponential dependence on the depth of the ... More