How Does Momentum Benefit Deep Neural Networks Architecture Design A Few Case Studies

We present and review an algorithmic and theoretical framework for improving neural network architecture design via momentum . We consider how momentum can improve the architecture design for recurrent neural networks, neural ordinary differential equations (ODEs), and transformers . Weshow that integrating momentum into neural network architectures has several remarkable theoretical and empirical benefits, including overcoming the vanishing gradient issues in training RNNs and neural ODEs, resulting in effective learning long-termdependencies .…

The Complexity of Bipartite Gaussian Boson Sampling

Gaussian boson sampling is a model of photonic quantum computing that has attracted attention as a platform for building quantum devices capable of performing tasks that are out of reach for classical devices . We show that, under the standard anti-concentration and Permanent-of-Gaussians conjectures, there is noefficient classical algorithm to sample from ideal Gaussian Boson samplingdistributions (even approximately) unless the polynomial hierarchy collapses .…

ADOP Approximate Differentiable One Pixel Point Rendering

We present a novel point-based, differentiable neural rendering pipeline for scene refinement and novel view synthesis . The point cloud rendering is performed by adifferentiable renderer using multi-resolution point rasterization . After rendering, the neural image pyramid is passed through a deep neural network for shading calculations and hole-filling .…

The algebra of row monomial matrices

The class of row monomial matrices is closed under multiplication, but not under ordinary matrix addition . The most significant difference is the summation operation . The algebra plays an important role in the study of DFA,especially for synchronizing automata .…

Towards Efficient NLP A Standard Evaluation and A Strong Baseline

The ELUE benchmark is publicly available athttp://eluebenchmark.fastnlp.top/. We demonstrate the ElasticBERT outperforms or performs on par with SOTA compressed and early exiting models . We alsopre-train and release a strong baseline, ElasticBERt, whose elasticity is bothstatic and dynamic . The benchmark is public and a public leaderboard for forefficient NLP models .…

Identification of Metallic Objects using Spectral Magnetic Polarizability Tensor Signatures Object Classification

Magnetic polarizability tensor (MPT) offers an economical characterisation of metallic objects . MPT spectral signature can be determined frommeasurements of the induced voltage over a range frequencies in a metalsignature for a hidden object . With classification in mind, it can also becomomomputed in advance for different threat and non-threat objects .…

Easy plane spin Hall nano oscillators as spiking neurons for neuromorphic computing

We show analytically using a macrospin approximation that easy-plane spinHall nano-oscillators excited by a spin-current have phase dynamics analogous to that of Josephson junctions . We simulate two elementary neuralnetwork blocks that implement operations essential for neuromorphic computing . First, we show that output spikes energies from two neurons can be summed and injected into a following layer neuron and second, we demonstrate that outputspikes can be multiplied by synaptic weights implemented by locally modifying theanisotropy .…

A Novel Clustering Based Algorithm for Continuous and Non invasive Cuff Less Blood Pressure Estimation

Continuous blood pressure (BP) measurements can reflect a bodys response todiseases and serve as a predictor of cardiovascular and other healthconditions . Current cuff-based BP measurement methods are incapable of providing continuous BP readings . The proposed clustering approach helps obtain more accurate estimates ofSystolic Blood pressure (SBP) and Diastolic Blood Pressure (DBP) Using the clustering .…

Variational and numerical analysis of a mathbf Q tensor model for smectic A liquid crystals

We analyse an energy minimisation problem recently proposed for modellingsmectic-A liquid crystals . The optimality conditions give a coupled nonlinearsystem of partial differential equations . Our two main results are aproof of the existence of solutions to the minimisation problems and a priori error estimates for its discretisation using the $\mathcal{C}^0$ interior penalty method .…

Object DGCNN 3D Object Detection using Dynamic Graphs

3D object detection often involves complicated training and testingpipelines, which require substantial domain knowledge about individualdatasets . Inspired by recent non-maximum suppression-free 2D objects detection models, we propose a new architecture on point clouds . This approach aligns the outputs of the teachermodel and the student model in a permutation-invariant fashion, significantly simplifying knowledge distillation for the 3D detection task .…

Bundle Networks Fiber Bundles Local Trivializations and a Generative Approach to Exploring Many to one Maps

Many-to-one maps are ubiquitous in machine learning . In many problems it is useful to explore, understand, and sample from a model’s fibers . In this paper we show that popular generativearchitectures are ill-suited to such tasks . Motivated by this we introduce anovel generative architecture, a Bundle Network, based on the concept of afiber bundle from (differential) topology .…

Ego4D Around the World in 3 000 Hours of Egocentric Video

Ego4D offers 3,025 hours of daily-life activity video spanning hundreds of scenarios (household, outdoor, workplace, leisure, etc.) captured by 855 uniquecamera wearers from 74 worldwide locations and 9 different countries . Portions of the video are accompanied by audio, audio, 3D meshes of the environment, eye gaze, stereo, and/or synchronized videos from multiple egocentric cameras at the same event .…

Interpretable AI forecasting for numerical relativity waveforms of quasi circular spinning non precessing binary black hole mergers

We present a deep-learning artificial intelligence model that is capable oflearning and forecasting the late-inspiral, merger and ringdown of numericalrelativity waveforms that describe quasi-circular, spinning, non-precessingbinary black hole mergers . We harnessed theThetaGPU supercomputer at the Argonne Leadership Computing Facility to trainour AI model using a training set of 1.5 million waveforms .…

Ousiometrics and Telegnomics The essence of meaning conforms to a two dimensional powerful weak and dangerous safe framework with diverse corpora presenting a safety bias

The essence of meaning conveyed by words is best described by a compass-like power-danger framework . Analysis of a disparate collection of large-scale English language corpora –literature, news, Wikipedia, talk radio, and social media — shows that naturallanguage exhibits a systematic bias toward safe, low danger words .…

SAR Net A Scenario Aware Ranking Network for PersonalizedFair Recommendation in Hundreds of Travel Scenarios

Alibaba serves an indispensable role forhundreds of different travel scenarios from Fliggy, Taobao, Alipay apps, etc. To provide personalized recommendation service for users visiting differentscenarios, there are two critical issues to be carefully addressed . In this paper, we propose a novelScenario-Aware Ranking Network (SAR-Net) to address these issues .…

AI Total Analyzing Security ML Models with Imperfect Data in Production

Development of new machine learning models is typically done on manuallycurated data sets, making them unsuitable for evaluating the models’ performance during operations . With this in mind, we developed a web-based visualization system that allows users to quickly gather headline performance numbers while maintaining confidence that the underlying data pipeline is functioning properly .…

HEDP A Method for Early Forecasting Software Defects based on Human Error Mechanisms

The main process behind a software defect is that an error-prone scenariotriggers human error modes, which psychologists have observed to recur acrossdiverse activities . The proposed ideaemphasizes predicting the exact location and form of a possible defect . The approach was able to predict, atthe requirement phase, the exact locations and forms of 7 out of the 22 (31.8%) specific types of defects that were found in the code .…

Compliance checking in reified IO logic via SHACL

This paper presents a methodology to carry out compliance checking on reifiedI/O logic formulae . These are translated in SHACL (Shapes Constraint Language)shapes, a recent W3C recommendation to validate and reason with RDFtriplestores . Compliance checking is then enforced by validating RDF graphs describing states of affairs with respect to these ShACL shapes .…

A Cross Platform Benchmark for Interval Computation Libraries

Interval computation widely used to certify computations that use floatingpoint operations to avoid pitfalls related to rounding error introduced byinaccurate operations . Despite its popularity and practical benefits, supportfor interval arithmetic is not standardized nor available in mainstamprogramming languages . We propose the first benchmark for intervalcomputations, coupled with reference solutions computed with exact arithmetic,and compare popular C and C++ libraries over different architectures, operatingsystems, and compilers .…

An algorithm for a fairer and better voting system

The major finding, of this article, is an ensemble method that aims to solve the problem of finding the best candidate to represent the voters . We have convincing evidence that our algorithm is better than Instant-RunoffVoting, Preferential Block Voting, Single Transferable Vote, and First Past ThePost .…

Dynamic Conflict Resolution of IoT Services in Smart Homes

We propose a novel conflict resolution framework for IoT services in multi-resident smart homes . The proposed framework employs a preferenceextraction model based on a temporal proximity strategy . We design a preferenceaggregation model using a matrix factorization-based approach . The concepts of current resident item matrix and idealresident item matrix are introduced as key criteria to cater to the conflictresolution framework .…

Improving the Robustness to Variations of Objects and Instructions with a Neuro Symbolic Approach for Interactive Instruction Following

An interactive instruction following task has been proposed as a benchmark for learning to map natural language instructions and first-person vision intosequences of actions to interact with objects in a 3D simulated environment . We assume that this problem is due to the high sensitiveness of neural feature extraction to small changes invision and language inputs .…

Machine Learning For Elliptic PDEs Fast Rate Generalization Bound Neural Scaling Law and Minimax Optimality

In this paper, we study the statistical limits of deep learning techniques for solving elliptic partial differential equations . We focus on a prototype elliptic PDE: theSchrodinger equation on a hypercube with zero Dirichlet boundary condition . We establishupper and lower bounds for both methods, which improves upon concurrentlydeveloped upper bounds for this problem via a fast rate generalization bound .…

Model hierarchies and higher order discretisation of time dependent thin film free boundary problems with dynamic contact angle

We present a mathematical and numerical framework for thin-film fluid flow . We provide algorithmic details and an implementation of higher-order spatial and temporal discretisation of the underlying free boundary problem using the finite element method . We investigate the impact of the dynamic contact angle on the evolution of two benchmark problems:gravity-driven sliding droplets and the instability of a ridge .…

Spectral theory for Maxwell s equations at the interface of a metamaterial Part II Limiting absorption limiting amplitude principles and interface resonance

This paper is concerned with the time-dependent Maxwell’s equations for aplane interface between a negative material described by the Drude model and the vacuum . In a first paper, we have constructed a generalized Fourier transform which diagonalizesthe Hamiltonian that represents the propagation of transverse electric waves .…

Block Contextual MDPs for Continual Learning

In reinforcement learning, the environment dynamics is implicitly assumed to be stationary . This assumption of stationarity, while simplifying, can be unrealistic in many scenarios . In this work, we propose to examinethis continual reinforcement learning setting through the block contextual MDP(BC-MDP) framework .…

Output Space Entropy Search Framework for Multi Objective Bayesian Optimization

We consider the problem of black-box multi-objective optimization (MOO) usingexpensive function evaluations (also referred to as experiments) The goal is to approximate the true Pareto set of solutions by minimizing the total resource cost of experiments . We appropriately instantiate the principle of OSEsearch to derive efficient algorithms for the following four MOO problemsettings: 1) The most basic em single-fidelity setting, where experiments areexpensive and accurate; 2) Handling em black-boxes constraints { which cannot beevaluated without performing experiments; 3) The discrete multi-financesetting, where .…

The springback penalty for robust signal recovery

We propose a new penalty for constructing models to recover an unknown signal from incomplete and inaccuratemeasurements . Mathematically, the springback penalty is a weakly convexfunction . It bears various theoretical and computational advantages of both the benchmark convex $ell_1 penalty and many of its non-convex surrogatesthat have been well studied in the literature .…

Representing Matrices Using Algebraic ZX calculus

Elementary matrices play an important role in linear algebra applications . We show their properties on inverses and transpose using rewriting rules of ZX-calculus . We are able to depict any matrices of size 2^m\times 2^n by string diagrams without resorting to a diagrammatic normal form for matrices as shown in [Wang 2020].…