Transfer Learning for Human & AI

Transformers do not scale very well to long sequence lengths largely because of quadratic self-attention complexity . To this date, there is no well-established consensus on how to evaluate this class of models . This paper proposes a systematic and unifiedbenchmark, LRA, specifically focused on evaluating model quality underlong-context scenarios .…

A function on an algebra is congruence preserving if it maps pairs of congruent elements onto pairs of congruent 7 elements . We show that on the algebra of binary trees whose leaves 8 are labeled by letters of an analphabet containing at least three letters, a 9 function is a polynomial .…

Deep learning-based models have been widely studied for click-through rate (CTR) prediction and lead to improved prediction accuracy in many industrial applications . KD is a teacher-student learning framework to transfer knowledge learned from a teacher model to a student model .…

We consider an entireclassification of these invariances called the twelvefold way in enumerativecombinatorics . We develop a method to characterize lossless compression limits . Explicit computations for all twelve settings are carried out for i.i.d.uniform and Bernoulli distributions . Comparisons among settings provide quantitative insight .…

In this letter, we investigate a generic model assessing the effect ofin-phase/quadrature-phase imbalance (IQI) on an asymmetric dual hop radiofrequency/free space optical (RF/FSO) two-way relay (TWR) system . The fading on the RF and FSO links have been modeled using K-distribution and doublegeneralized Gamma (D-GG) turbulence model, respectively .…

When training automated systems, it has been shown to be beneficial to adapt the representation of data by learning a problem-specific metric . We extend this idea and, for the widely used family of k nearestneighbors algorithms, develop a method that allows learning locally adaptivemetrics .…

Dy-ERNIE is a non-Euclidean embedding approach that learns evolving entityrepresentations in a product of Riemannian manifolds . This approach is better than existing embedding approaches for temporal KGs, such as hierarchical and cyclic structures . We let the entity representations evolve according to a velocity vector defined in the tangent space at each timestamp .…

We propose to incorporate emotion as prior for the probabilistic state-of-the-art textgeneration model such as GPT-2 . The model gives a user the flexibility to control the category and intensity of emotion as well as the topic of the generated text .…

The problem of finding the optimal beamformers in massive multiple-input multiple- input multiple-output (MIMO) networks is challenging because of its non-convexity . This paper proposes a noveldeep learning based beamforming algorithm to address the above challenges . Numerical results based on both synthetic and ray-tracing channel models show that the proposed neural network can achieve high WSRs with significantly reduced runtime, while exhibiting favorable generalization capability with respect to the antenna number, BS number and the inter-BS distance .…

Face-to-face conversation in Virtual Reality (VR) is a challenge when participants wear head-mounted displays (HMD) A significant portion of aparticipant’s face is hidden and facial expressions are difficult to perceive . We propose one of the first low-cost systems for this task which uses only open source, free software and affordable hardware .…

MultipleTopologies Prediction (MTP) is a data-driven trajectory-prediction mechanism . MTP outperforms a state-of-the-art multimodaltrajectory prediction baseline (MFP) in terms of prediction accuracy by 78.24% on a challenging simulated dataset . We show that MTP enables ouroptimization-based planner, MTPnav, to achieve collision-free and time-efficient navigation across a variety of challenging intersectionscenarios on the CARLA simulator .…

Bayesian inference is a principled framework for dealing with uncertainty . Robust Optimisation Monte Carlo (ROMC) is one of the most recent techniques of the specific domain . Our implementation provides arobust and efficient solution to a practitioner who wants to perform inference on a simulator-based model .…

Model-based planning is often thought to be necessary for deep, carefulreasoning and generalization in artificial agents . We study theperformance of MuZero (Schrittwieser et al., 2019), a state-of-the-art MBRL algorithm, under a number of interventions and ablations . Our resultssuggests the primary benefit of planning is in driving policylearning .…

The trade-off between differential privacy and statistical accuracy in ingeneralized linear models (GLMs) is studied . We propose differentially private algorithms for parameter estimation in both low-dimensional and high-dimensional sparse GLMs and characterize their statistical performance . Weestablish privacy-constrained minimax lower bounds for GLMs, which imply that the proposed algorithms are rate-optimal up to logarithmic factors in samplesize .…

Many real-world systems, such as moving planets, can be considered asmulti-agent dynamic systems . LG-ODE is a latentordinary differential equation generative model for modeling multi-agentdynamic system with known graph structure . It can simultaneously learn theembedding of high dimensional trajectories and infer continuous latent systemdynamics .…

The measurement of treatment (intervention) effects on a single (or just afew) treated unit(s) based on counterfactuals constructed from artificialcontrols has become a popular practice in applied statistics and economic . In high-dimensionalsetting, we often use principal component or (weakly) sparse regression toestimate counterfactsuals .…

In a sequential decision-making problem, off-policy evaluation (OPE)estimates the expected cumulative reward of a target policy using loggedtransition data generated from a different behavior policy, without execution of the target policy . Reinforcement learning in high-stake environments, such as healthcare and education, is often limited to off-Policy settings due to safety or ethical concerns, or inability of exploration .…

The existing Neural ODE formulation relies on an explicit knowledge of the termination time . We extend neural ODEs to implicitly defined terminationcriteria modeled by neural event functions . We proposeimulation-based training of point processes with applications in discretecontrol. We testour approach in modeling hybrid discrete- and continuous- systems such asswitching dynamical systems and collision in multi-body systems, and we proposesimulation based training ofpoint processes .…

Navigation is one of the most widely used applications of the Location BasedServices (LBS) This paper proposes and implements a novel path finding algorithm for open areas, i.e.areas with no network of pathways such as grasslands and parks . The proposed algorithm creates a new graph in the open area, which can consider the obstacles and barriers and calculate the path based on the factors that are important for wheelchair users .…

In autonomous navigation of mobile robots, sensors suffer from massiveocclusion in cluttered environments . Humans infer the exact shape of the obstacles from only partial observation andgenerate non-conservative trajectories that avoid possible collisions inoccluded space . Method leverages the performance of a kinodynamic planner by improving security with no reduction of speed in clustered environments, authors say .…

We consider the cooperative multi-player version of the stochasticmulti-armed bandit problem . We study the regime where the players cannotcommunicate but have access to shared randomness . In prior work by the firsttwo authors, a strategy for this regime was constructed for two players and three arms .…

Federated Learning enables multiple clients to collaboratively learn a machinelearning model while keeping all their data on-device . Communication between the clients and the server is considered a main bottleneck in the convergence time of federated learning . Adaptive Federated Dropout (AFD) is a novel technique to reduce the communication costs associated with federatedlearning .…

This paper provides a statistical analysis of high-dimensional batchReinforcement Learning (RL) using sparse linear function approximation . It sheds light on thefact that sparsity-aware methods can make batch RL more sample efficient . The results suggest that having well-conditioned data iscrucial for sparse batch policy learning .…

Chatbots are emerging as a promising platform for accessing and delivering healthcare services . Chatbots aim to take an active role in the provision of prevention, diagnosis, and treatment services . This article takes a closer look at how these emerging chatbots address design aspects relevant to healthcareservice provision, emphasizing the Human-AI interaction aspects .…

This paper tackles the problem of the simultaneous interference among themultiple users in the downlink of a wireless multiantenna system . The paper also proposes a new transmission technique that jointly optimizes the transmit symbol-levelprecoding and the constellation rotation of the data stream for each user .…

Monte Carlo methods act as a convenient bridge for connectingintractable mathematical expressions with actionable estimates via sampling . Instead of focusing on distributions, we articulate Gaussian conditionals at the level of randomvariables . We show how this interpretation of conditioning gives riseto a general family of approximations that lend themselves to fast sampling from Gaussian process posteriors.…

Convolutional Neural Networks can be designed with different levels of complexity depending upon the task at hand . This paper analyzes the effect ofdimensional changes to the CNN architecture on its performance on the task ofHistopathological Cancer Classification . The research starts with a baseline10-layer CNN model with (3 X 3) convolution filters .…

Recommendation systems underpin the serving of nearly all online content inthe modern age . From Youtube and Netflix recommendations, to Facebook feeds andGoogle searches, these systems are designed to filter content to the predictedpreferences of users . These systems have faced growing criticism withrespect to their impact on content diversity, social polarization, and the health of public discourse .…

Motivated by the lack of systematic tools to obtain safe control laws for hybrid systems, we propose an optimization-based framework . We assume a setting in which the system dynamics are known and in which data exhibiting safe systembehavior is available .…

Thompson sampling has been shown to be an effective policy across a variety of online learning tasks . We prove an asymptotic convergence result for Thompson sampling under the assumption of a sub-linear Bayesian regret, and show that the actions of a Thompson sampling agent provide a stronglyconsistent estimator of the optimal action .…

We show that by controlling parts of a physical environment in which a deep neural network is being fine-tuned online, an adversary can launch subtle data poisoning attacks that degrade the performance of the system . While the attack can be applied in general to any perception task, weconsider a DNN based traffic light classifier for an autonomous car that has been trained in one city and is fine-tuneed online in another city .…

While recurrent models have been effective in NLP tasks, their performance on context-free languages (CFLs) has been found to be quite weak . Given that CFLsare believed to capture important phenomena such as hierarchical structure innatural languages, this discrepancy in performance calls for an explanation .…

FairLens is a methodology for discovering and explaining biases in AI systems . It can be used to audit a fictional commercial black-box model acting as a clinical decision support system . FairLens allows experts to investigate whether to trust the model and to spotlight group-specific biases that might constitutepotential fairness issues .…

How to use the multi-modal image features moreefficiently is still a challenging problem in the field of medical imagesegmentation . In this paper, we develop a cross-Modal self-attentiondistillation network by exploiting the encoded information of the intermediate layers from different modalities .…

Distance metric learning has attracted much attention in recent years . Real-world applications often involveattributed sequence data (e.g., clickstreams), where each instance consists ofnot only a set of attributes but also a sequence of categorical items . We propose a deep learning framework, called MLAS (Metric Learningon Attributed Sequences), to learn a distance metric that effectively measures differences between attributed sequences .…

The deep neural networks (DNNs) have achieved great success in learning complex patterns with strong predictive power, but they are often thought of as “black box” models without sufficient transparency andinterpretability . This paper aims to unwrap the black box of deepReLU networks through local linear representation .…

Distributed energy resources (DERs) are playing an increasing role inancillary services for the bulk grid, particularly in frequency regulation . The proposed framework is hierarchical, consisting of a toplayer and a bottom layer . The bottom layer consists of the DERs inside each microgrid whose power levels are adjusted so that the output of the corresponding aggregator in the top layer matches the output in the bottom layer.…

This manuscript presents the formulation, implementation, calibration andapplication of a multiscale reduced-order model to simulate a titanium panelstructure subjected to thermo-mechanical loading associated with high-speedflight . The proposed approach was fully calibrated using a series ofuniaxial tension tests of Ti-6242S at a wide range of temperatures and twodifferent strain rates .…

A systematic procedure to synthesize interval observers for nonlineardiscrete-time systems is proposed . The feedback gains and other matrices are found from the solutions to semidefinite feasibility programs . Two cases are considered: (1) the interval observer is in the same coordinate frame as the given system, and (2) the intervals observer uses a coordinate transformation .…

The Hierarchical Chinese Postman Problem is finding a shortest traversal of all edges of a graph respecting precedence constraints given by a partial order on classes of edges . We show that the special case with connected classes isNP-hard even on orders decomposable into a chain and an incomparable class .…

Due to structural and functional abnormalities or genetic variations andmutations, there may be dysfunctional molecules within an intracellularsignaling network that do not allow the network to correctly regulate its output molecules . This disruption in signalinginterrupts normal cellular functions and may eventually develop somepathological conditions .…

Decentralized electricity markets are often dominated by a small set ofgenerator companies who control the majority of the capacity . We find that capacity has an impact on the average electricity price in asingle year . This work can helpinform policy on how to best regulate a market to ensure that the price ofelectricity remains competitive .…

The semiconductor and IC industry is facing the issue of high energy consumption . By utilizing stochastic computing, we can achieve higher energy efficiency and smaller area sizes in terms of designing arithmetic units . Also, we aim to popularize the affiliation ofStochastic systems in designing futuristic BLSI and Neuromorphic systems.…

In theoretical computer science, the tiling problem is intimately to the study of parallel repetition theorems . Kindler et al.\ showed that for general bodies $B$ this is tight, i.e.\ that there is a tiling body of $\mathbb{R}^n$ whosesurface area is $O(n/\sqrt{n)$ The result suggests that while strong parallel repetition fails in general, there may be important special cases where it still applies .…

The integration of renewables into electrical grids calls foroptimization-based control schemes requiring reliable grid models . The presentwork proposes a method for simultaneously minimizing grid operation cost and estimating line parameters based on methods for the optimal design of experiments .…

The optimal tracking problem is addressed in robotics literature by using a variety of robust and adaptive control approaches . This scheme minimizes the tracking errors and optimizes the overall dynamical behavior using simultaneouslinear feedback control strategies . Reinforcement learning approaches based onvalue iteration processes are adopted to solve the underlying Bellmanoptimality equations .…

We give a Markov chain based algorithm for sampling almost uniform solutions of constraint satisfaction problems (CSPs) We give the current best almost-uniform samplers for hypergraphcolorings and for CNF solutions . Our main approach is a new technique called states compression, which generalizes the”mark/unmark” paradigm of Moitra (Moitra, JACM, 2019), and can give fastlocal-lemma-based sampling algorithms .…

The main design principles in computer architecture have recently shifted from a monolithic scaling-driven approach to the development of heterogeneousarchitectures that tightly co-integrate multiple specialized processor andmemory chiplets . In such data-hungry multi-chip architectures, current networks-in-Package (NiPs) may not be enough to cater to their fast-changing communication demands .…