total 119174took 0.13s

Moderate-Deviations of Lossy Source Coding for Discrete and Gaussian SourcesNov 09 2011May 10 2012We study the moderate-deviations (MD) setting for lossy source coding of stationary memoryless sources. More specifically, we derive fundamental compression limits of source codes whose rates are $R(D) \pm \epsilon_n$, where $R(D)$ is the rate-distortion ... More

Asymptotic Estimates in Information Theory with Non-Vanishing Error ProbabilitiesApr 10 2015This monograph presents a unified treatment of single- and multi-user problems in Shannon's information theory where we depart from the requirement that the error probability decays asymptotically in the blocklength. Instead, the error probabilities for ... More

On the Reliability Function of the Discrete Memoryless Relay ChannelApr 12 2013Dec 23 2014Bounds on the reliability function for the discrete memoryless relay channel are derived using the method of types. Two achievable error exponents are derived based on partial decode-forward and compress-forward which are well-known superposition block-Markov ... More

A Formula for the Capacity of the General Gel'fand-Pinsker ChannelOct 03 2012Apr 16 2014We consider the Gel'fand-Pinsker problem in which the channel and state are general, i.e., possibly non-stationary, non-memoryless and non-ergodic. Using the information spectrum method and a non-trivial modification of the piggyback coding lemma by Wyner, ... More

On Dispersions of Discrete Memoryless Channels with Noncausal State Information at the EncoderApr 02 2012Nov 12 2013In this paper, we study the finite blocklength limits of state-dependent discrete memoryless channels where the discrete memoryless state is known noncausally at the encoder. For the point-to-point case, this is known as the Gel'fand-Pinsker channel model. ... More

Error and Erasure Exponents for the Broadcast Channel with Degraded Message SetsJan 27 2015Error and erasure exponents for the broadcast channel with degraded message sets are analyzed. The focus of our error probability analysis is on the main receiver where, nominally, both messages are to be decoded. A two-step decoding algorithm is proposed ... More

Analysis of Optimization Algorithms via Sum-of-SquaresJun 11 2019In this work, we introduce a new framework for unifying and systematizing the performance analysis of first-order black-box optimization algorithms for unconstrained convex minimization over finite-dimensional Euclidean spaces. The low-cost iteration ... More

Asymptotic Expansions for Gaussian Channels with Feedback under a Peak Power ConstraintOct 09 2014Oct 13 2014This paper investigates the asymptotic expansion for the size of block codes defined for the additive white Gaussian noise (AWGN) channel with feedback under the following setting: A peak power constraint is imposed on every transmitted codeword, and ... More

Strong Converse for Discrete Memoryless Networks with Tight Cut-Set BoundsJun 15 2016Jun 17 2016This paper proves the strong converse for any discrete memoryless network (DMN) with tight cut-set bound, i.e., whose cut-set bound is achievable. Our result implies that for any DMN with tight cut-set bound and any fixed rate tuple that resides outside ... More

Empirical Output Distribution of Good Delay-Limited Codes for Quasi-Static Fading ChannelsOct 29 2015Nov 14 2016This paper considers delay-limited communication over quasi-static fading channels under a long-term power constraint. A sequence of length-$n$ delay-limited codes for a quasi-static fading channel is said to be capacity-achieving if the codes achieve ... More

On AWGN Channels and Gaussian MACs with Variable-Length FeedbackSep 02 2016Sep 09 2016We characterize the information-theoretic limits of the additive white Gaussian noise (AWGN) channel and the Gaussian multiple access channel (MAC) when variable-length feedback is available at the encoder and a non-vanishing error probability is permitted. ... More

Analysis of Remaining Uncertainties and Exponents under Various Conditional Rényi EntropiesMay 31 2016In this paper, we analyze the asymptotics of the normalized remaining uncertainty of a source when a compressed or hashed version of it and correlated side-information is observed. For this system, commonly known as Slepian-Wolf source coding, we establish ... More

A Unified Convergence Analysis of the Multiplicative Update Algorithm for Nonnegative Matrix FactorizationSep 04 2016Sep 06 2016The multiplicative update (MU) algorithm has been used extensively to estimate the basis and coefficient matrices in nonnegative matrix factorization (NMF) problems under a wide range of divergences and regularizations. However, theoretical convergence ... More

Minimum Rates of Approximate Sufficient StatisticsDec 08 2016Given a sufficient statistic for a parametric family of distributions, one can estimate the parameter without access to the data itself. However, the memory or code size for storing the sufficient statistic may nonetheless still be prohibitive. Indeed, ... More

Rényi Resolvability and Its Applications to the Wiretap ChannelJul 04 2017Dec 03 2018The conventional channel resolvability problem refers to the determination of the minimum rate required for an input process so that the output distribution approximates a target distribution in either the total variation distance or the relative entropy. ... More

Asymptotic Coupling and Its Applications in Information TheoryDec 19 2017Jul 17 2018A coupling of two distributions $P_{X}$ and $P_{Y}$ is a joint distribution $P_{XY}$ with marginal distributions equal to $P_{X}$ and $P_{Y}$. Given marginals $P_{X}$ and $P_{Y}$ and a real-valued function $f$ of the joint distribution $P_{XY}$, what ... More

Corrections to "Wyner's Common Information under Rényi Divergence Measures"Oct 05 2018In this correspondence, we correct an erroneous argument in the proof of Theorem 1 of the paper above, which is a statement generalizing that for Wyner's common information.

Bounds on the Average Distance and Distance Enumerator with Applications to Non-Interactive SimulationApr 08 2019Apr 16 2019We leverage proof techniques in coding theory and Fourier analysis to derive new bounds for the problem of non-interactive simulation of random variables. Previous bounds in the literature were derived by applying data processing inequalities concerning ... More

A Proof of the Strong Converse Theorem for Gaussian Multiple Access ChannelsFeb 11 2015Apr 26 2016We prove the strong converse for the $N$-source Gaussian multiple access channel (MAC). In particular, we show that any rate tuple that can be supported by a sequence of codes with asymptotic average error probability less than one must lie in the Cover-Wyner ... More

Strong Converse Theorems for Classes of Multimessage Multicast Networks: A Rényi Divergence ApproachJul 09 2014Jul 31 2015This paper establishes that the strong converse holds for some classes of discrete memoryless multimessage multicast networks (DM-MMNs) whose corresponding cut-set bounds are tight, i.e., coincide with the set of achievable rate tuples. The strong converse ... More

On the Scaling Exponent of Polar Codes for Binary-Input Energy-Harvesting ChannelsJan 06 2016Apr 26 2016This paper investigates the scaling exponent of polar codes for binary-input energy-harvesting (EH) channels with infinite-capacity batteries. The EH process is characterized by a sequence of i.i.d. random variables with finite variances. The scaling ... More

Second-Order Asymptotics for the Discrete Memoryless MAC with Degraded Message SetsAug 05 2014Apr 14 2015This paper studies the second-order asymptotics of the discrete memoryless multiple-access channel with degraded message sets. For a fixed average error probability $\epsilon\in(0,1)$ and an arbitrary point on the boundary of the capacity region, we characterize ... More

Second-Order Asymptotics for the Classical Capacity of Image-Additive Quantum ChannelsAug 29 2013Feb 17 2015We study non-asymptotic fundamental limits for transmitting classical information over memoryless quantum channels, i.e. we investigate the amount of classical information that can be transmitted when a quantum channel is used a finite number of times ... More

Fixed Error Asymptotics For Erasure and List DecodingFeb 20 2014Apr 21 2014We derive the optimum second-order coding rates, known as second-order capacities, for erasure and list decoding. For erasure decoding for discrete memoryless channels, we show that second-order capacity is $\sqrt{V}\Phi^{-1}(\epsilon_t)$ where $V$ is ... More

Empirical Output Distribution of Good Delay-Limited Codes for Quasi-Static Fading ChannelsOct 29 2015Nov 26 2015This paper considers delay-limited communication over quasi-static fading channels under a long-term power constraint. A sequence of length-$n$ delay-limited codes for a quasi-static fading channel is said to be capacity-achieving if the codes achieve ... More

Automatic Relevance Determination in Nonnegative Matrix Factorization with the β-DivergenceNov 25 2011Oct 05 2012This paper addresses the estimation of the latent dimensionality in nonnegative matrix factorization (NMF) with the \beta-divergence. The \beta-divergence is a family of cost functions that includes the squared Euclidean distance, Kullback-Leibler and ... More

Achievable Rates for Gaussian Degraded Relay Channels with Non-Vanishing Error ProbabilitiesApr 06 2016Oct 18 2016This paper revisits the Gaussian degraded relay channel, where the link that carries information from the source to the destination is a physically degraded version of the link that carries information from the source to the relay. The source and the ... More

Improved Bounds on Sidon Sets via Lattice Packings of SimplicesOct 05 2016Nov 17 2016A $ B_h $ set (or Sidon set of order $ h $) in an Abelian group $ G $ is any subset $ \{b_0, b_1, \ldots,b_{n}\} \subset G $ with the property that all the sums $ b_{i_1} + \cdots + b_{i_h} $ are different up to the order of the summands. Let $ \phi(h,n) ... More

On Gaussian MACs with Variable-Length Feedback and Non-Vanishing Error~ProbabilitiesSep 02 2016Jan 09 2018We characterize the fundamental limits of transmission of information over a Gaussian multiple access channel (MAC) with the use of variable-length feedback codes and under a non-vanishing error probability formalism. We develop new achievability and ... More

Information Spectrum Approach to Strong Converse Theorems for Degraded Wiretap ChannelsJun 26 2014Sep 26 2014We consider block codes for degraded wiretap channels in which the legitimate receiver decodes the message with an asymptotic error probability no larger than $\varepsilon$ but the leakage to the eavesdropper vanishes. For discrete memoryless and Gaussian ... More

Wyner's Common Information under Rényi Divergence MeasuresSep 07 2017Jan 25 2018We study a generalized version of Wyner's common information problem (also coined the distributed sources simulation problem). The original common information problem consists in understanding the minimum rate of the common input to independent processors ... More

On Exact and $\infty$-Rényi Common InformationsSep 30 2018Oct 15 2018Recently, two extensions of Wyner's common information \textemdash{} exact and R\'enyi common informations \textemdash{} were introduced respectively by Kumar, Li, and El Gamal (KLE), and the present authors. The class of common information problems refers ... More

On Exact and $\infty$-Rényi Common InformationsSep 30 2018May 15 2019Recently, two extensions of Wyner's common information \textemdash{} exact and R\'enyi common informations \textemdash{} were introduced respectively by Kumar, Li, and El Gamal (KLE), and the present authors. The class of common information problems refers ... More

Exact Error and Erasure Exponents for the Asymmetric Broadcast ChannelJan 16 2018Nov 09 2018We derive exact (ensemble-tight) error and erasure exponents for the asymmetric broadcast channel given a random superposition codebook. We consider Forney's optimal decoder for both messages and the message pair for the receiver that decodes both messages. ... More

The Reliability Function of Lossy Source-Channel Coding of Variable-Length Codes with FeedbackOct 20 2017Apr 07 2019We consider transmission of discrete memoryless sources (DMSes) across discrete memoryless channels (DMCs) using variable-length lossy source-channel codes with feedback. The reliability function (optimum error exponent) is shown to be equal to $\max\{0, ... More

A Unified Convergence Analysis of the Multiplicative Update Algorithm for Regularized Nonnegative Matrix FactorizationSep 04 2016Jun 07 2017The multiplicative update (MU) algorithm has been extensively used to estimate the basis and coefficient matrices in nonnegative matrix factorization (NMF) problems under a wide range of divergences and regularizers. However, theoretical convergence guarantees ... More

Online Nonnegative Matrix Factorization with OutliersApr 10 2016Oct 15 2016We propose a unified and systematic framework for performing online nonnegative matrix factorization in the presence of outliers. Our framework is particularly suited to large-scale data. We propose two solvers based on projected gradient descent and ... More

Improved Bounds on Sidon Sets via Lattice Packings of SimplicesOct 05 2016Sep 28 2017A $ B_h $ set (or Sidon set of order $ h $) in an Abelian group $ G $ is any subset $ \{b_0, b_1, \ldots,b_{n}\} $ of $ G $ with the property that all the sums $ b_{i_1} + \cdots + b_{i_h} $ are different up to the order of the summands. Let $ \phi(h,n) ... More

Online Nonnegative Matrix Factorization with OutliersApr 10 2016We propose a unified and systematic framework for performing online nonnegative matrix factorization in the presence of outliers that is particularly suited to large datasets. Within this framework, we propose two solvers based on proximal gradient descent ... More

The Third-Order Term in the Normal Approximation for the AWGN ChannelNov 11 2013Nov 19 2013This paper shows that, under the average error probability formalism, the third-order term in the normal approximation for the additive white Gaussian noise channel with a maximal or equal power constraint is at least $\frac{1}{2} \log n + O(1)$. This ... More

Second-Order Coding Rates for Channels with StateMay 29 2013May 14 2014We study the performance limits of state-dependent discrete memoryless channels with a discrete state available at both the encoder and the decoder. We establish the epsilon-capacity as well as necessary and sufficient conditions for the strong converse ... More

Estimating Signals with Finite Rate of Innovation from Noisy Samples: A Stochastic AlgorithmJan 01 2008Jan 03 2008As an example of the recently-introduced concept of rate of innovation, signals that are linear combinations of a finite number of Diracs per unit time can be acquired by linear filtering followed by uniform sampling. However, in reality, samples are ... More

Second-Order Asymptotics for the Gaussian MAC with Degraded Message SetsOct 04 2013Oct 06 2015This paper studies the second-order asymptotics of the Gaussian multiple-access channel with degraded message sets. For a fixed average error probability $\varepsilon \in (0,1)$ and an arbitrary point on the boundary of the capacity region, we characterize ... More

Equivocations, Exponents and Second-Order Coding Rates under Various Rényi Information MeasuresApr 10 2015Jul 05 2016We evaluate the asymptotics of equivocations, their exponents as well as their second-order coding rates under various R\'{e}nyi information measures. Specifically, we consider the effect of applying a hash function on a source and we quantify the level ... More

Moderate Deviation Asymptotics for Variable-Length Codes with FeedbackJul 16 2017May 16 2018We consider data transmission across discrete memoryless channels (DMCs) using variable-length codes with feedback. We consider the family of such codes whose rates are $\rho_N$ below the channel capacity $C$, where $\rho_N$ is a positive sequence that ... More

Bounds on the Average Distance and Distance Enumerator with Applications to Non-Interactive SimulationApr 08 2019We leverage proof techniques in coding theory and Fourier analysis to derive new bounds for the problem of non-interactive simulation of random variables. Previous bounds in the literature were derived by applying data processing inequalities concerning ... More

A Numerical Study on the Wiretap Network with a Simple Network TopologyMay 12 2015Jan 15 2016In this paper, we study a security problem on a simple wiretap network, consisting of a source node S, a destination node D, and an intermediate node R. The intermediate node connects the source and the destination nodes via a set of noiseless parallel ... More

Time-Division is Optimal for Covert Communication over Some Broadcast ChannelsOct 26 2017Oct 10 2018We consider a covert communication scenario where a transmitter wishes to communicate simultaneously to two legitimate receivers while ensuring that the communication is not detected by an adversary, the warden. The legitimate receivers and the adversary ... More

Simulation of Random Variables under Rényi Divergence Measures of All OrdersMay 31 2018Dec 21 2018The random variable simulation problem consists in using a $k$-dimensional i.i.d. random vector $X^{k}$ with distribution $P_{X}^{k}$ to simulate an $n$-dimensional i.i.d. random vector $Y^{n}$ so that its distribution is approximately $Q_{Y}^{n}$. In ... More

Sidon Sets of Fixed Cardinality and Lattice-Packings of SimplicesOct 05 2016Oct 08 2016A $ B_h $ set (or Sidon set of order $ h $) in an Abelian group $ G $ is any subset $ \{b_0, b_1, \ldots,b_{n}\} \subset G $ with the property that all the sums $ b_{i_1} + \cdots + b_{i_h} $ are different up to the order of the summands. Let $ \phi(h,n) ... More

A Tight Upper Bound for the Third-Order Asymptotics for Most Discrete Memoryless ChannelsDec 15 2012Oct 24 2013This paper shows that the logarithm of the epsilon-error capacity (average error probability) for n uses of a discrete memoryless channel is upper bounded by the normal approximation plus a third-order term that does not exceed 1/2 log n + O(1) if the ... More

On the Dispersions of Three Network Information Theory ProblemsJan 18 2012Nov 13 2013We analyze the dispersions of distributed lossless source coding (the Slepian-Wolf problem), the multiple-access channel and the asymmetric broadcast channel. For the two-encoder Slepian-Wolf problem, we introduce a quantity known as the entropy dispersion ... More

Asymmetric Evaluations of Erasure and Undetected Error ProbabilitiesJul 01 2014Oct 21 2015The problem of channel coding with the erasure option is revisited for discrete memoryless channels. The interplay between the code rate, the undetected and total error probabilities is characterized. Using the information spectrum method, a sequence ... More

On the Scaling Exponent of Polar Codes for Binary-Input Energy-Harvesting ChannelsJan 06 2016Nov 14 2016This paper investigates the scaling exponent of polar codes for binary-input energy-harvesting (EH) channels with infinite-capacity batteries. The EH process is characterized by a sequence of i.i.d. random variables with finite variances. The scaling ... More

A Proof of the Strong Converse Theorem for Gaussian Broadcast Channels via the Gaussian Poincaré InequalitySep 04 2015Sep 11 2015We prove that 2-user Gaussian broadcast channels admit the strong converse. This implies that every sequence of block codes with an asymptotic average error probability smaller than one is such that all the limit points of the sequence of rate pairs must ... More

On the Reliability Function of the Common-Message Broadcast Channel with Variable-Length FeedbackJan 06 2017Jan 12 2017We derive upper and lower bounds on the reliability function for the common-message discrete memoryless broadcast channel with variable-length feedback. We show that the bounds are tight when the broadcast channel is stochastically degraded. For the achievability ... More

The Optimal Compression Rate of Variable-to-Fixed Length Source Coding with a Non-Vanishing Excess-Distortion ProbabilityMar 19 2018We consider the variable-to-fixed length lossy source coding (VFSC) problem. The optimal compression rate of the average length of variable-to-fixed source coding, allowing a non-vanishing probability of excess-distortion $\varepsilon$, is shown to be ... More

Exact Channel SynthesisOct 30 2018We consider the exact channel synthesis problem, which is the problem of determining how much information is required to create exact correlation remotely when there is a certain amount of randomness shared by two terminals. This problem generalizes an ... More

Codes in the Space of Multisets---Coding for Permutation Channels with ImpairmentsDec 28 2016Jun 21 2018Motivated by communication channels in which the transmitted sequences are subject to random permutations, as well as by certain DNA storage systems, we study the error control problem in settings where the information is stored/transmitted in the form ... More

Asymptotically Optimal Codes Correcting Fixed-Length Duplication Errors in DNA Storage SystemsAug 30 2018A (tandem) duplication of length $ k $ is an insertion of an exact copy of a substring of length $ k $ next to its original position. This and related types of impairments are of relevance in modeling communication in the presence of synchronization errors, ... More

Exact Error and Erasure Exponents for the Asymmetric Broadcast ChannelJan 16 2018Jun 07 2019Consider the asymmetric broadcast channel with a random superposition codebook, which may be comprised of constant composition or \iid codewords. By applying Forney's optimal decoder for individual messages and the message pair for the receiver that decodes ... More

Unequal Message Protection: Asymptotic and Non-Asymptotic TradeoffsMay 05 2014We study a form of unequal error protection that we term "unequal message protection" (UMP). The message set of a UMP code is a union of $m$ disjoint message classes. Each class has its own error protection requirement, with some classes needing better ... More

Rank Minimization over Finite Fields: Fundamental Limits and Coding-Theoretic InterpretationsApr 21 2011Dec 01 2011This paper establishes information-theoretic limits in estimating a finite field low-rank matrix given random linear measurements of it. These linear measurements are obtained by taking inner products of the low-rank matrix with random sensing matrices. ... More

Zero-Error Shift-Correcting and Shift-Detecting CodesMay 09 2016Motivated by communication scenarios such as timing channels (in queuing systems, molecular communications, etc.) and bit-shift channels (in magnetic recording systems), we study the error control problem in cases where the dominant type of noise are ... More

Second-Order and Moderate Deviation Asymptotics for Successive RefinementJan 18 2016Aug 26 2016We derive the optimal second-order coding region and moderate deviations constant for successive refinement source coding with a joint excess-distortion probability constraint. We consider two scenarios: (i) a discrete memoryless source (DMS) and arbitrary ... More

High-Dimensional Gaussian Graphical Model Selection: Walk Summability and Local Separation CriterionJul 06 2011Mar 04 2012We consider the problem of high-dimensional Gaussian graphical model selection. We identify a set of graphs for which an efficient estimation algorithm exists, and this algorithm is based on thresholding of empirical conditional covariances. Under a set ... More

Learning Gaussian Tree Models: Analysis of Error Exponents and Extremal StructuresSep 28 2009Jan 05 2010The problem of learning tree-structured Gaussian graphical models from independent and identically distributed (i.i.d.) samples is considered. The influence of the tree structure and the parameters of the Gaussian distribution on the learning rate as ... More

The Sender-Excited Secret Key Agreement Model: Capacity, Reliability and Secrecy ExponentsJul 21 2011Oct 10 2013We consider the secret key generation problem when sources are randomly excited by the sender and there is a noiseless public discussion channel. Our setting is thus similar to recent works on channels with action-dependent states where the channel state ... More

The Dispersion of Mismatched Joint Source-Channel Coding for Arbitrary Sources and Additive ChannelsNov 30 2017Aug 31 2018We consider a joint source channel coding (JSCC) problem in which we desire to transmit an arbitrary memoryless source over an arbitrary additive channel. We propose a mismatched coding architecture that consists of Gaussian codebooks for both the source ... More

Online Nonnegative Matrix Factorization with General DivergencesJul 30 2016Aug 02 2016We develop a unified and systematic framework for performing online nonnegative matrix factorization under a wide variety of important divergences. The online nature of our algorithm makes it particularly amenable to large-scale data. We prove that the ... More

Learning High-Dimensional Markov Forest Distributions: Analysis of Error RatesMay 05 2010Feb 13 2011The problem of learning forest-structured discrete graphical models from i.i.d. samples is considered. An algorithm based on pruning of the Chow-Liu tree through adaptive thresholding is proposed. It is shown that this algorithm is both structurally consistent ... More

Discrete Lossy Gray-Wyner Revisited: Second-Order Asymptotics, Large and Moderate DeviationsDec 03 2015Jul 18 2016In this paper, we revisit the discrete lossy Gray-Wyner problem. In particular, we derive its optimal second-order coding rate region, its error exponent (reliability function) and its moderate deviations constant under mild conditions on the source. ... More

Refined Asymptotics for Rate-Distortion using Gaussian Codebooks for Arbitrary SourcesAug 16 2017Aug 31 2018The rate-distortion saddle-point problem considered by Lapidoth (1997) consists in finding the minimum rate to compress an arbitrary ergodic source when one is constrained to use a random Gaussian codebook and minimum (Euclidean) distance encoding is ... More

Second- and Third-Order Asymptotics of the Continuous-Time Poisson ChannelMar 25 2019Apr 08 2019The paper derives the optimal second- and third-order coding rates for the continuous-time Poisson channel. This is the first instance of a second-order result for a continuous-time channel. The converse proof hinges on a novel construction of an output ... More

Fundamental Limits of Communication Over State-Dependent Channels With FeedbackDec 21 2017Jan 15 2019The fundamental limits of communication over state-dependent discrete memoryless channels with noiseless feedback are studied, under the assumption that the communicating parties are allowed to use variable-length coding schemes. Various cases are analyzed, ... More

Lower Bounds on the Bayes Risk of the Bayesian BTL Model with Applications to Comparison GraphsSep 27 2017Apr 20 2018We consider the problem of aggregating pairwise comparisons to obtain a consensus ranking order over a collection of objects. We use the popular Bradley-Terry-Luce (BTL) model which allows us to probabilistically describe pairwise comparisons between ... More

Thompson Sampling for Cascading BanditsOct 02 2018We design and analyze TS-Cascade, a Thompson sampling algorithm for the cascading bandit problem. In TS-Cascade, Bayesian estimates of the click probability are constructed using a univariate Gaussian; this leads to a more efficient exploration procedure ... More

On the Throughput of Channels that Wear OutJan 05 2017Aug 06 2018This work investigates the fundamental limits of communication over a noisy discrete memoryless channel that wears out, in the sense of signal-dependent catastrophic failure. In particular, we consider a channel that starts as a memoryless binary-input ... More

A Ranking Model Motivated by Nonnegative Matrix Factorization with Applications to Tennis TournamentsMar 15 2019We propose a novel ranking model that combines the Bradley-Terry-Luce probability model with a nonnegative matrix factorization framework to model and uncover the presence of latent variables that influence the performance of top tennis players. We derive ... More

Strong Converse for Hypothesis Testing Against Independence over a Two-Hop NetworkAug 16 2018By proving a strong converse, we strengthen the weak converse result by Salehkalaibar, Wigger and Wang (2017) concerning hypothesis testing against independence over a two-hop network with communication constraints. Our proof follows by judiciously combining ... More

The Dispersion of Nearest-Neighbor Decoding for Additive Non-Gaussian ChannelsDec 21 2015Oct 19 2016We study the second-order asymptotics of information transmission using random Gaussian codebooks and nearest neighbor (NN) decoding over a power-limited stationary memoryless additive non-Gaussian noise channel. We show that the dispersion term depends ... More

Second-Order Asymptotics of the Continuous-Time Poisson ChannelMar 25 2019The paper derives the optimal second-order coding rate for the continuous-time Poisson channel. This is the first instance of a second-order result for a continuous-time channel. The converse proof hinges on a novel construction of an output distribution ... More

Distributed Detection with Empirically Observed StatisticsMar 14 2019We consider a binary distributed detection problem in which the distributions of the sensor observations are unknown and only empirically observed statistics are available to the fusion center. The source (test) sequences are transmitted through different ... More

Second-Order Coding Rates for Conditional Rate-DistortionOct 10 2014This paper characterizes the second-order coding rates for lossy source coding with side information available at both the encoder and the decoder. We first provide non-asymptotic bounds for this problem and then specialize the non-asymptotic bounds for ... More

Moderate Deviations Asymptotics for Streaming Compression of Correlated SourcesApr 25 2016In this paper, we consider the problem of blockwise streaming compression of a pair of correlated sources, which we term streaming Slepian-Wolf coding. We study the moderate deviations regime in which the rate pairs of a sequence of codes converges, along ... More

Canonical Estimation in a Rare-Events RegimeSep 21 2011Oct 05 2011We propose a general methodology for performing statistical inference within a `rare-events regime' that was recently suggested by Wagner, Viswanath and Kulkarni. Our approach allows one to easily establish consistent estimators for a very large class ... More

Second-Order Asymptotics of the Continuous-Time Poisson ChannelMar 25 2019Apr 03 2019The paper derives the optimal second-order coding rate for the continuous-time Poisson channel. This is the first instance of a second-order result for a continuous-time channel. The converse proof hinges on a novel construction of an output distribution ... More

On Gaussian Channels with Feedback under Expected Power Constraints and with Non-Vanishing Error ProbabilitiesDec 16 2015Sep 21 2016In this paper, we consider single- and multi-user Gaussian channels with feedback under expected power constraints and with non-vanishing error probabilities. In the first of two contributions, we study asymptotic expansions for the additive white Gaussian ... More

Non-Asymptotic and Second-Order Achievability Bounds for Coding With Side-InformationJan 28 2013Dec 25 2014We present novel non-asymptotic or finite blocklength achievability bounds for three side-information problems in network information theory. These include (i) the Wyner-Ahlswede-Korner (WAK) problem of almost-lossless source coding with rate-limited ... More

Zero-Error Capacity of $P$-ary Shift Channels and FIFO QueuesMay 09 2016Sep 10 2017The objects of study of this paper are communication channels in which the dominant type of noise are symbol shifts, the main motivating examples being timing and bit-shift channels. Two channel models are introduced and their zero-error capacities and ... More

Streaming Data Transmission in the Moderate Deviations and Central Limit RegimesDec 19 2015We consider streaming data transmission over a discrete memoryless channel. A new message is given to the encoder at the beginning of each block and the decoder decodes each message sequentially, after a delay of $T$ blocks. In this streaming setup, we ... More

Second-Order Asymptotically Optimal Statistical ClassificationJun 03 2018Dec 06 2018Motivated by real-world machine learning applications, we analyze approximations to the non-asymptotic fundamental limits of statistical classification. In the binary version of this problem, given two training sequences generated according to two {\em ... More

Error Exponent Bounds for the Bee-Identification ProblemMay 20 2019Jun 04 2019Consider the problem of identifying a massive number of bees, uniquely labeled with barcodes, using noisy measurements. We formally introduce this `bee-identification problem', define its error exponent, and derive efficiently computable upper and lower ... More

A Case Where Interference Does Not Affect The Channel DispersionApr 01 2014In 1975, Carleial presented a special case of an interference channel in which the interference does not reduce the capacity of the constituent point-to-point Gaussian channels. In this work, we show that if the inequalities in the conditions that Carleial ... More

Exponential Strong Converse for Content Identification with Lossy RecoveryFeb 22 2017Dec 25 2017We revisit the high-dimensional content identification with lossy recovery problem (Tuncel and G\"und\"uz, 2014) and establish an exponential strong converse theorem. As a corollary of the exponential strong converse theorem, we derive an upper bound ... More

Non-Asymptotic Achievable Rates for Energy-Harvesting Channels using Save-and-TransmitJul 09 2015Apr 26 2016This paper investigates the information-theoretic limits of energy-harvesting (EH) channels in the finite blocklength regime. The EH process is characterized by a sequence of i.i.d. random variables with finite variances. We use the save-and-transmit ... More

Distributed Detection with Empirically Observed StatisticsMar 14 2019May 04 2019Consider a distributed detection problem in which the underlying distributions of the observations are unknown; instead of these distributions, noisy versions of empirically observed statistics are available to the fusion center. These empirically observed ... More

Exact Moderate Deviation Asymptotics in Streaming Data TransmissionApr 23 2016In this paper, a streaming transmission setup is considered where an encoder observes a new message in the beginning of each block and a decoder sequentially decodes each message after a delay of $T$ blocks. In this streaming setup, the fundamental interplay ... More

Error Exponent Bounds for the Bee-Identification ProblemMay 20 2019Consider the problem of identifying a massive number of bees, uniquely labeled with barcodes, using noisy measurements. We formally introduce this ``bee-identification problem'', define its error exponent, and derive efficiently computable upper and lower ... More

Stochastic L-BFGS: Improved Convergence Rates and Practical Acceleration StrategiesApr 01 2017Oct 24 2017We revisit the stochastic limited-memory BFGS (L-BFGS) algorithm. By proposing a new framework for the convergence analysis, we prove improved convergence rates and computational complexities of the stochastic L-BFGS algorithms compared to previous works. ... More