Learning to Approximate Functions Using Nb doped SrTiO _3 Memristors

Memristors have attracted interest as neuromorphic computation elements because they show promise in enabling efficient hardware implementations of artificial neurons and synapses . We performed measurements on interface-typememristors to validate their use in neuromorphic hardware . Using this class of memristivedevices as the synaptic weight element in a spiking neural network yields, toour knowledge, one of the first models of this kind, capable of learning to be a universal function approximator, and strongly suggests the suitability of these memristors for usage in future computing platforms .…

Low Complexity Models for Acoustic Scene Classification Based on Receptive Field Regularization and Frequency Damping

Deep Neural Networks are known to be very demanding in terms of computing and memory requirements . Using our proposed filter filterdamping, we achieved the 1st rank at the DCASE-2020 Challenge in the task of low-Complexity Acoustic Scene Classification . We propose afilter-damping technique for regularizing the RF of models, without altering their architecture and changing their parameter counts.…

Multi class Spectral Clustering with Overlaps for Speaker Diarization

This paper describes a method for overlap-aware speaker diarization . Given anoverlap detector and a speaker embedding extractor, our method performsspectral clustering of segments informed by the output of the overlap detector . Our method achieves a testdiarization error rate (DER) of 24.0% on the mixed-headset setting of the AMImeeting corpus, which is a relative improvement of 15.2% over a strongagglomerative hierarchical clustering baseline .…

Mixed Nondeterministic Probabilistic Interfaces

Interface theories are powerful frameworks supporting incremental andcompositional design of systems through refinements and constructs forconjunction, and parallel composition . In this report we present a first Interface Theor — |Modal Mixed Interfaces — for systems exhibiting bothnon-determinism and randomness in their behaviour .…

Multi Accent Adaptation based on Gate Mechanism

When only a limited amount of accented speech data is available, the conventional approach isaccent-specific adaptation . To simplify the adaptation procedure, we exploreadapting the baseline model to multiple target accents simultaneously withmulti-accent mixed data . We propose using accent-specific top layer withgate mechanism (AST-G) to realize multi-acccent adaptation .…

Deep Neural Network Fingerprinting by Conferrable Adversarial Examples

In Machine Learning as a Service, a provider trains a deep neural network and provides many users access . The hosted (source) model is susceptible to model stealing attacks, where an adversary derives a \emph{surrogate model} from API access to the source model… For post hoc detection of such attacks, the provider needs a robust method to determine whether a suspect model is a surrogate of their model .…

BW EDA EEND Streaming End to End Neural Speaker Diarization for a Variable Number of Speakers

Online end-to-end neural diarization system, BW-EDA-EEND, processes data incrementally for a variable number of speakers . System is based on the EDA architecture of Horiguchi et al., but utilizes theincremental Transformer encoder, attending only to its left contexts and usingblock-level recurrence in the hidden states to carry information from block to block, making the algorithm complexity linear in time .…

A Comparison Study on Infant Parent Voice Diarization

We design a framework for studying prelinguistic child voice from 3 to 24months based on state-of-the-art algorithms in di-arization . Our system consists of a time-invariant feature ex-tractor, a context-dependent embeddinggenerator, and a clas-sifier . We found that our best system achieved 43.8% DER ontestdataset, compared to 55.4% DERN achieved by LENA soft-ware .…

An SMT Based Approach for Verifying Binarized Neural Networks

Deep learning has emerged as an effective approach for creating modern software systems, with neural networks often surpassing hand-crafted systems . Formal verification is a promising avenue for tackling this difficulty, by formally certifying that networks are correct . We propose anSMT-based technique for verifying \emph{binarized neural networks} – a popularkind of neural networks, where some weights have been binarized in order to make the network more memory and energy efficient, and quicker toevaluate .…

Latent Causal Invariant Model

Current supervised learning can learn spurious correlation during the data-fitting process, imposing issues regarding interpretability,out-of-distribution (OOD) generalization, and robustness . To avoid spurious correlation, we propose a Latent Causal Invariance Model (LaCIM) which pursuescausal prediction . We introduce latent variables that areseparated into (a) output-causative factors and (b) others that are spuriouslycorrelated to the output via confounders, to model the underlying causalfactors .…

Residual Likelihood Forests

ResidualLikelihood Forests (RLF) produces conditional likelihoods that are sequentially optimized using global loss in the context of previouslearners within a boosting-like framework . This increases the efficiency of our strong classifier, allowing for the design of classifiers which are more compact in terms of model capacity .…

Against Adversarial Learning Naturally Distinguish Known and Unknown in Open Set Domain Adaptation

Open set domain adaptation refers to the scenario that the target domaincontains categories that do not exist in the source domain . It is a more commonsituation in the reality compared with the typical closed set domains adaptation . We propose an “againstadversarial learning” method that can distinguish unknown target data and knowndata naturally without setting any additional hyper parameters and the target data predicted to the known classes can be classified at the same time .Experimental…

Optimizing Transformer for Low Resource Neural Machine Translation

Language pairs with limited amounts of parallel data remain a challenge for neural machine translation . Using an optimized Transformer for low-resource conditions improves the translation quality up to 7.3 BLEU points compared to using the Transformer default settings . Our experiments on different subsets of the IWSLT14 training data show that theeffectiveness of Transformer under low-Resource conditions is highly dependent on the hyper-parameter settings .…

Concentration Inequalities for Statistical Inference

This paper gives a review of concentration inequalities which are widelyemployed in analyzes of mathematical statistics in a wide range of settings . From sub-Gaussian to sub-exponential, sub-Weibull random variables, we aim to illustrate theconcentration inequalities with known constants and to improve existing boundswith sharper constants .…

A deep learning classifier for local ancestry inference

Local ancestry inference (LAI) identifies the ancestry of each segment of an individual’s genome and is an important step in medical and population geneticstudies . We formulate the LAItask as an image segmentation problem and develop a new LAI tool using a deepconvolutional neural network with an encoder-decoder architecture .…

Capped norm linear discriminant analysis and its applications

Classical linear discriminant analysis (LDA) is based on squared Frobeniousnorm and hence is sensitive to outliers and noise . To improve the robustness of LDA, we introduce capped l_{2,1}-norm of a matrix, which employsnon-squared l_2-norm and “capped” operation, we propose a novel cappedl_{2,.1}norm linear discriminative analysis .…

Learning to Rank with Missing Data via Generative Adversarial Networks

We explore role of Conditional Generative Adversarial Networks (GAN) inimputing missing data . We apply GAN imputation on a novel use case ine-commerce: a learning-to-rank problem with incomplete training data . Using an AmazonSearch ranking dataset, we produce standard ranking models trained onGAN-imputed data that are comparable to training on ground-truth data based on NDCG and MRR .…

Can We Trust Deep Speech Prior

Speech enhancement (SE) based on deep speech prior has attracted attention, such as the variational auto-encoder with non-negative matrixfactorization (VAE-NMF) architecture . Despite the clear advantage in theory, we argue that deep priors must be used with much caution, since the likelihood produced by a deep generative model does not always coincide with the speech quality .…

Mixed Set Domain Adaptation

In the settings of conventional domain adaptation, categories of the sourcedataset are from the same domain, which is not always true in reality . In this paper, we propose a mixed set domain adaptation method that can reduce distribution discrepancy between different categories .…

Moving Forward in Formation A Decentralized Hierarchical Learning Approach to Multi Agent Moving Together

Previous multi-agent pathfinding (MAPF) methods hardly take formation into consideration . Other decentralized partially observable approaches to MAPF arereinforcement learning (RL) methods . However, these RL methods encounterdifficulties when learning path finding and formation problem at the same time . In this paper, we propose a novel algorithm that uses a hierarchical structure to decompose the multi objective task into unrelated ones .…

EAdam Optimizer How ε Impact Adam

Many adaptive optimization methods have been proposed and used in deeplearning . Adam is regarded as the default algorithm and widely used in many deep learning frameworks . Recently, many variants of Adam, such asAdabound, RAdam and Adabelief, show better performancethan Adam .…

Toward Force Estimation in Robot Assisted Surgery using Deep Learning with Vision and Robot State

Vision-based deep learning using convolutional neural networks is apromising approach for providing useful force estimates . The network with both state and vision inputs outperformed aphysics-based baseline model in accuracy . It showed comparable accuracy butfaster computation times than a baseline recurrent neural network, making itbetter suited for real-time applications.…

Reverse engineering learned optimizers reveals known and novel mechanisms

Learned optimizers are algorithms that can themselves be trained to solve optimization problems . In contrast to baseline optimizers (such as momentum orAdam) that use simple update rules, learnedoptimizers use flexible, high-dimensional, nonlinear parameterizations . We study learned optimizers trained from scratch on threedisparate tasks, and discover that they have learned interpretable mechanisms, including: momentum, gradient clipping, learning rate schedules, and a new form of learning rate adaptation .…

RetroXpert Decompose Retrosynthesis Prediction like a Chemist

Retrosynthesis is the process of recursively decomposing target molecules into available building blocks . It plays an important role in solving problems in organic synthesis planning . The method disassembles retrosyntheses into twosteps: i. identify the potential reaction center of the target molecule through a novel graph neural network and generate intermediate synthons .…

Hyperspectral classification of blood like substances using machine learning methods combined with genetic algorithms in transductive and inductive scenarios

This study is focused on applying genetic algorithms (GA) to model and bandselection in hyperspectral image classification . We use a forensic-inspired data set of seven hyperspectrals images with blood and five visually similarsubstances to test GA-optimised classifiers in two scenarios: when the trainingand test data come from the same image and when they come from different images, which is a more challenging task due to significant spectradifferences .…

On Self Distilling Graph Neural Network

The teacher-student knowledge distillation framework hasdemonstrated its potential in training Graph Neural Networks (GNNs) However,due to the difficulty of training deep and wide GNN models, one can not alwaysobtain a satisfactory teacher model for distillation . We propose the first GNNSelf-Distillation framework for GNNs, termed GNN self-distillation, that serves as a drop-in replacement for improving the training process .…

Do Noises Bother Human and Neural Networks In the Same Way A Medical Image Analysis Perspective

Deep learning had already demonstrated its power in medical images, including denoising, classification, segmentation, etc. All these applications are proposed to automatically analyze medical images beforehand, which brings more information to radiologists during clinical assessment for accuracyimprovement . In experiments, we apply the proposed framework to differentdatasets, models, and use cases .…

Correlation based Multi phasal models for improved imagined speech EEG recognition

Translation of imagined speech electroencephalogram(EEG) into humanunderstandable commands greatly facilitates the design of naturalistic braincomputer interfaces . To achieve improved imagined speech unit classification,this work aims to profit from the parallel information contained in multi-phasal EEG data recorded while speaking, imagining and performingarticulatory movements corresponding to specific speech units .…

Hybrid Supervised Reinforced Model for Dialogue Systems

This paper presents a recurrent hybrid model and training procedure fortask-oriented dialogue systems based on Deep Recurrent Q-Networks (DRQN) The model copes with both tasks required for Dialogue Management: State Trackingand Decision Making . It is based on modeling Human-Machine interaction into alatent representation embedding an interaction context to guide the discussion .…