Dot Product

Processing in memory (PIM) moves computation into memories with the goal of improving throughput and energy-efficiency compared to traditional von Neumann-based architectures. Most existing PIM architectures are either generalpurpose but only support atomistic operations, or are specialized to accelerate a single task. We propose the Parallel Processor in Associative Content-addressable memory (PPAC), a novel in-memory accelerator that supports a range of matrix-vectorproduct (MVP)-like operations that find use in traditional and emerging applications. PPAC is, for example, able to accelerate low-precision neural networks, exact/approximate hash lookups, cryptography, and forward error correction. The fully-digital nature of PPAC enables its implementation with standard-cellbased CMOS, which facilitates automated design and portability among technology nodes. To demonstrate the efficacy of PPAC, we provide post-layout implementation results in 28nm CMOS for different array sizes. A comparison with recent digital and mixed-signal PIM accelerators reveals that PPAC is competitive in terms of throughput and energy-efficiency, while accelerating a wide range of applications and simplifying development.

In this paper, we study dot-product sets and k-simplices in Z d n for odd n, where Z n is the ring of residues modulo n. We show that if E is sufficiently large then the dotproduct set of E covers the whole ring. In higher dimensional cases, if E is sufficiently large then the set of simplices and the set of dot-product simplices determined by E, up to congurence, have positive densities.

We study a family of variants of Erdős' unit distance problem, concerning distances and dot products between pairs of points chosen from a large finite point set. Specifically, given a large finite set of n points E, we look for bounds on how many subsets of k points satisfy a set of relationships between point pairs based on distances or dot products. We survey some of the recent work in the area and present several new, more general families of bounds.

We study a family of variants of Erdős' unit distance problem, concerning distances and dot products between pairs of points chosen from a large finite point set. Specifically, given a large finite set of n points E, we look for bounds on how many subsets of k points satisfy a set of relationships between point pairs based on distances or dot products. We survey some of the recent work in the area and present several new, more general families of bounds.

The approximation capability of support vector machines (SVMs) is investigated. We show the universal approximation capability of SVMs with various kernels, including Gaussian, several dot product, or polynomial kernels, based on the universal approximation capability of their standard feedforward neural network counterparts. Moreover, it is shown that an SVM with polynomial kernel of degree p À 1 which is trained on a training set of size p can approximate the p training points up to any accuracy.

We connect two random graph models, the Popularity Adjusted Block Model (PABM) and the Generalized Random Dot Product Graph (GRDPG), by demonstrating that the PABM is a special case of the GRDPG in which communities correspond to mutually orthogonal subspaces of latent vectors. This insight allows us to construct new algorithms for community detection and parameter estimation for the PABM, as well as improve an existing algorithm that relies on Sparse Subspace Clustering. Using established asymptotic properties of Adjacency Spectral Embedding for the GRDPG, we derive asymptotic properties of these algorithms. In particular, we demonstrate that the absolute number of community detection errors tends to zero as the number of graph vertices tends to infinity. Simulation experiments illustrate these properties.

In Physics, many quantities are vectors, and their use requires typical operations such as addition, subtraction, scalar multiplication, scalar product (dot product), vector product (cross product), and scalar triple product. This is a very basic topic in all General Physics courses for Engineering degrees. However, we have detected that some students lack a deep understanding of vector operations and their properties. In this study, we present a virtual laboratory (developed using the tool “Easy Java Simulations”) for the study and understanding of these topics. The user can introduce the components of the input vectors and gets a three‐dimensional representation, which can be scaled and rotated for better visualization. Any of the aforementioned operations can be selected, and the result is shown both numerically and graphically. The user can also modify any represented vector. In this way, the virtual lab provides a real‐time visualization of how the change affects the result. Th...

Data classification is one of the most fundamental tasks that can be accomplished by supervised machine learning. There exists a lot of algorithms, and they have the specific case of uses. Different classification methods follow different techniques to map the relationship between input and output. This article proposes an angle measurement-based classification technique called Support Directional Shifting Vectors (SDSV) to segment a spectral domain into regions with a very effective solution for classification problems. This method introduces two shifting vectors, named Support Direction Vector (SDV) and Support Origin Vector (SOV). These vectors are formed as a linear function to measure cosine-angle by the dot product of two separated data classes, named target data points and non-target data points. Considering the target class samples, the vectors get aligned in a way that the angle with the target class gets minimized, while the angle with the non-target class gets maximized. The error in the position of the linear function has been modeled as the loss function. Then, the vector position is updated iteratively by optimizing this loss function using a gradient descent algorithm. We have used this model to classify data from two different machine learning datasets to evaluate the performance of the proposed method. Finally, we have carefully examined the results and compared them with the other standard classification algorithms. In summary, the proposed SDSV algorithm has shown a comparable accuracy compared to different standard algorithms.

Saccadic localization of spatially extended objects requires the computation of a single saccadic landing position. What representation of the target guides saccades? Saccades were examined for various targets composed of dots to determine whether landing position corresponded to the center-of-gravity (average location) of the dots, the center-of-area of the shape, or the symmetric axis. Targets were composed of dots configured as outline drawings of circles, ellipses, cardioids, wiggly lines, or amorphous blobs. In some cases, dot spacing was varied, extraneous dot clusters were superimposed, or different distributions of dots inside the boundary were added. Quasi-random dot clusters without a well-defined contour were also studied. Instructions were to look at the target as a whole, and keep latency long enough to avoid compromising accuracy. Saccades landed with a high level of precision (S.D.s 7-10% of target eccentricity) near the center-of-area of the target shape, rather than at the center-of-gravity of the dots or on the symmetric axis. Landing position was unaffected by the spacing of dots along the boundary, the addition of dots within the boundary, or the addition of the extraneous dot clusters. When the target was a cluster of quasi-random dots, saccades landed closer to the center-of-area of the implied surface than to the average location of the dots. Overall, the positions of individual dots were important only insofar as the dots affected overall target shape. The results show that a representation of target shape guides saccades, rather than a more primitive representation of individual elements within the attended region.

We present an efficient document representation learning framework, Document Vector through Corruption (Doc2VecC). Doc2VecC represents each document as a simple average of word embeddings. It ensures a representation generated as such captures the semantic meanings of the document during learning. A corruption model is included, which introduces a data-dependent regularization that favors informative or rare words while forcing the embeddings of common and non-discriminative ones to be close to zero. Doc2VecC produces significantly better word embeddings than Word2Vec. We compare Doc2VecC with several state-of-the-art document representation learning algorithms. The simple model architecture introduced by Doc2VecC matches or out-performs the state-of-the-art in generating high-quality document representations for sentiment analysis, document classification as well as semantic relatedness tasks. The simplicity of the model enables training on billions of words per hour on a single machine. At the same time, the model is very efficient in generating representations of unseen documents at test time.

Random graphs are increasingly becoming objects of interest for modeling networks in a wide range of applications. Latent position random graph models posit that each node is associated with a latent position vector, and that these vectors follow some geometric structure in the latent space. In this paper, we consider random dot product graphs, in which an edge is formed between two nodes with probability given by the inner product of their respective latent positions. We assume that the latent position vectors lie on an unknown one-dimensional curve and are coupled with a response covariate via a regression model. Using the geometry of the underlying latent position vectors, we propose a manifold learning and graph embedding technique to predict the response variable on out-of-sample nodes, and we establish convergence guarantees for these responses. Our theoretical results are supported by simulations and an application to Drosophila brain data.

Several different techniques and softwares intend to improve the accuracy of results computed in a fixed finite precision. Here we focus on a method to improve the accuracy of polynomial evaluation via Horner's scheme. Such an algorithm exists for polynomials with real floating point coefficients. In this paper, we provide a new algorithm which deals with polynomials with complex floating point coefficients. We show that the computed result is as accurate as if computed in twice the working precision. The algorithm is simple since it only requires addition, subtraction and multiplication of floating point numbers in the same working precision as the given data. Such an algorithm can be useful for example to compute zeros of polynomial by Newton-like methods.

Several different techniques and softwares intend to improve the accuracy of results computed in a fixed finite precision. Here we focus on methods to improve the accuracy of summation, dot product and polynomial evaluation. Such algorithms exist real floating point numbers. In this paper, we provide new algorithms which deal with complex floating point numbers. We show that the computed results are as accurate as if computed in twice the working precision. The algorithms are simple since they only require addition, subtraction and multiplication of floating point numbers in the same working precision as the given data.

Several different techniques and softwares intend to improve the accuracy of results computed in a fixed finite precision. Here we focus on a method to improve the accuracy of polynomial evaluation via Horner's scheme. Such an algorithm exists for polynomials with real floating point coefficients. In this paper, we provide a new algorithm which deals with polynomials with complex floating point coefficients. We show that the computed result is as accurate as if com-puted in twice the working precision. The algorithm is simple since it only requires addition, subtraction and multiplication of floating point numbers in the same working precision as the given data. Such an algorithm can be useful for example to compute zeros of polynomial by Newton-like methods.

Several different techniques and softwares intend to improve the accuracy of results computed in a fixed finite precision. Here we focus on methods to improve the accuracy of summation, dot product and polynomial evaluation. Such algorithms exist real floating point numbers. In this paper, we provide new algorithms which deal with complex floating point numbers. We show that the computed results are as accurate as if computed in twice the working precision. The algorithms are simple since they only require addition, subtraction and multiplication of floating point numbers in the same working precision as the given data.

We explore variants of Erdős' unit distance problem concerning dot products between successive pairs of points chosen from a large finite subset of either F d q or Z d q , where q is a power of an odd prime. Specifically, given a large finite set of points E, and a sequence of elements of the base field (or ring) (α1,. .. , α k), we give conditions guaranteeing the expected number of (k+1)-tuples of distinct points (x1,. .. , x k+1) ∈ E k+1 satisfying xj • xj+1 = αj for every 1 ≤ j ≤ k.

In this paper we give necessary and sufficient conditions under which kernels of dot product type k(x, y) = k(x . y) satisfy Mercer's condition and thus may be used in Support Vector Machines (SVM), Regularization Networks (RN) or Gaussian Processes (GP). In particular, we show that if the kernel is analytic (i.e. can be expanded in a Taylor series), all expansion coefficients have to be nonnegative. We give an explicit functional form for the feature map by calculating its eigenfunctions and eigenvalues.

Faced with saturation of Moore's law and increasing size and dimension of data, system designers have increasingly resorted to parallel and distributed computing to reduce computation time of machine-learning algorithms. However, distributed computing is often bottle necked by a small fraction of slow processors called "stragglers" that reduce the speed of computation because the fusion node has to wait for all processors to complete their processing. To combat the effect of stragglers, recent literature proposes introducing redundancy in computations across processors, e.g., using repetition-based strategies or erasure codes. The fusion node can exploit this redundancy by completing the computation using outputs from only a subset of the processors, ignoring the stragglers. In this paper, we propose a novel technique-that we call "Short-Dot"-to introduce redundant computations in a coding theory inspired fashion, for computing linear transforms of long vectors. Instead of computing long dot products as required in the original linear transform, we construct a larger number of redundant and short dot products that can be computed faster and more efficiently at individual processors. In reference to comparable schemes that introduce redundancy to tackle stragglers, Short-Dot reduces the cost of computation, storage and communication since shorter portions are stored and computed at each processor, and also shorter portions of the input is communicated to each processor. Further, only a subset of these short dot products are required at the fusion node to finish the computation successfully, thus enabling us to ignore stragglers. We demonstrate through probabilistic analysis as well as experiments on computing clusters that Short-Dot offers significant speedup compared to existing techniques. We also derive trade-offs between the length of the dot-products and the resilience to stragglers (number of processors to wait for), for any such strategy and compare it to that achieved by our strategy. This work proposes a coding-theory inspired computation technique for speeding up computing linear transforms of high-dimensional data by distributing it across multiple processing units that compute shorter dot products. Our main focus is on addressing the "straggler effect," i.e., the problem of delays caused by a few slow processors that bottleneck the entire computation. To address this problem, we provide techniques (building on [1] [2] [3] [4] [5]) that introduce redundancy in the computation by designing a novel error-correction mechanism that allows the size of individual dot products computed at each processor to be shorter than the length of the input. Shorter dot products offer advantages in computation, storage and communication in distributed linear transforms. The problem of computing linear transforms of high-dimensional vectors is "the" critical step [6] in several machine learning and signal processing applications. Dimensionality reduction techniques such as Principal Component Analysis (PCA), Linear Discriminant Analysis (LDA), taking random

In this paper, we study dot-product sets and k-simplices in Z d n for odd n, where Z n is the ring of residues modulo n. We show that if E is sufficiently large then the dotproduct set of E covers the whole ring. In higher dimensional cases, if E is sufficiently large then the set of simplices and the set of dot-product simplices determined by E, up to congurence, have positive densities.

The existing work on distributed secure multi-party computation, e.g., set operations, dot product, ranking, focus on the privacy protection aspects, while the verifiability of user inputs and outcomes are neglected. Most of the existing works assume that the involved parties will follow the protocol honestly. In practice, a malicious adversary can easily forge his/her input values to achieve incorrect outcomes or simply lie about the computation results to cheat other parities. In this work, we focus on the problem of verifiable privacy preserving multiparty computation. We thoroughly analyze the attacks on existing privacy preserving multi-party computation approaches and design a series of protocols for dot product, ranging and ranking, which are proved to be privacy preserving and verifiable. We implement our protocols on laptops and mobile phones. The results show that our verifiable private computation protocols are efficient both in computation and communication.

This paper is focused on designing two parallel dot product implementations for heterogeneous master-worker platforms. These implementations are based on the data allocation and dynamic load balancing strategies. The first implementation is the dynamic master-worker with allocation of vectors where the master distributes vectors (data) and computations to the workers whereas the second one is the dynamic master-worker with allocation of vector pointers where the vectors are supposed to be replicated among participating resources beforehand and the master distributes computations to the workers. We also report a set of numerical experiments on a heterogeneous platform where computational resources have different computing powers. Also, the workers are connected to the master by links of same capacities. The obtained experimental results demonstrate that the dynamic allocation of vector pointers achieve better performance than the original one for computing dot product computation. The paper also presents and verifies an accurate timing model to predict the performance of the proposed implementations on clusters of heterogeneous workstations. Through this model the viability of the proposed implementa

In document-level sentiment classification, each document must be mapped to a fixed length vector. Document embedding models map each document to a dense, lowdimensional vector in continuous vector space. This paper proposes training document embeddings using cosine similarity instead of dot product. Experiments on the IMDB dataset show that accuracy is improved when using cosine similarity compared to using dot product, while using feature combination with Naïve Bayes weighted bag of n-grams achieves a new state of the art accuracy of 97.42%.

This work introduces the Least Dot Products (LDP) method, a new algorithm for phase stability analysis of thermodynamic mixtures. Starting with the fact that tangent plane distance function (the objective function used in global stability tests) is, essentially, the dot product between the vector of compositions of a trial phase and the vector that describes the differences among the chemical potential of the components in the phases, our approach tries to obtain the least values of an auxiliary function that represents an appropriate dot product on a unitary sphere of the n-dimensional space, which is a good approximation for the stability test function. In agreement with the foundations of the Gibbs plane tangent criterion, the new algorithm simply tries to find points where the objective function is negative, which are not (necessarily) stationary points or global minima. Thus, our main contribution is not a new method for general nonlinear problems, or a rule-based termination criteria for a classical optimization method. Consequently, if such points do not exist, then LDP method is not capable to recognize the stability condition. To overcome this problem, we develop also a powerful safeguard algorithm, denominated Projected Simulated Annealing (PJSA) algorithm, which is obtained by projecting the Simulated Annealing algorithm onto the unitary sphere. In the present article, firstly we consider liquid-liquid equilibria at low or moderate pressures, where the excess Gibbs energy is described by NRTL or UNIQUAC models. Secondly, we address vaporliquid equilibria at high pressures with cubic equations of state. To illustrate the performance of the new methodology, we use here 20 systems studied by other authors. Such problems possess 2-12 variables and constitute severe tests for many optimization methods. For some mixtures, we show that LDP method is capable of determining the instability condition using just two iterations. & 2011 Elsevier Ltd. All rights reserved. : ð2Þ If TPD(x)Z0, for all x A S, then the mixture is stable, and exhibits a single phase; otherwise, i.e., if there exists some x n A S such that TPD(x n) o0, then the mixture is unstable, and a new phase will appear. Michelsen (1982a) was the first to formulate numerical methodologies for the global stability analysis based on GPT criterion.

This work presents an energy-efficient SRAM with embedded dot-product computation capability, for binary-weight convolutional neural networks. A 10T bit-cell based SRAM array is used to store the 1-b filter weights. The array implements dotproduct as a weighted average of the bit-line voltages, which are proportional to the digital input values. Local integrating ADCs compute the digital convolution outputs, corresponding to each filter. We have successfully demonstrated functionality (> 98% accuracy) with the 10,000 test images in the MNIST handwritten digit recognition dataset, using 6-b inputs/outputs. Compared to conventional full-digital implementations using small bit-widths, we achieve similar or better energy-efficiency, by reducing data transfer, due to the highly parallel in-memory analog computations.

In metabolomics, metabolites from biological samples are measured by mass spectrometry (MS) and then identified by database searches for similar spectra. Since spectra from the same compound may differ depending on measurement conditions, comparison measure is a crucial issue for identifying compounds from their spectra. A standard method for comparison is normalized dot product or cosine correlation between spectral peaks. We discuss the weighting scheme for preprocessing peak intensities and demonstrate comparison results for about 9,000 spectra in our publicly available MassBank database ().

Data classification is one of the most fundamental tasks that can be accomplished by supervised machine learning. There exists a lot of algorithms, and they have the specific case of uses. Different classification methods follow different techniques to map the relationship between input and output. This article proposes an angle measurement-based classification technique called Support Directional Shifting Vectors (SDSV) to segment a spectral domain into regions with a very effective solution for classification problems. This method introduces two shifting vectors, named Support Direction Vector (SDV) and Support Origin Vector (SOV). These vectors are formed as a linear function to measure cosine-angle by the dot product of two separated data classes, named target data points and non-target data points. Considering the target class samples, the vectors get aligned in a way that the angle with the target class gets minimized, while the angle with the non-target class gets maximized. The error in the position of the linear function has been modeled as the loss function. Then, the vector position is updated iteratively by optimizing this loss function using a gradient descent algorithm. We have used this model to classify data from two different machine learning datasets to evaluate the performance of the proposed method. Finally, we have carefully examined the results and compared them with the other standard classification algorithms. In summary, the proposed SDSV algorithm has shown a comparable accuracy compared to different standard algorithms.

The primary and secondary school educational system should be stable and any upgrading reforms should be made gradually and consistently. This is especially important in mathematics education, since the element of logical reasoning while learning is more prominent there. Inconsistencies in reforms generate deficiencies in the higher levels of young students’ reasoning skills and this situation continues on the university stage of education. We will report our findings about the reasoning of first-year university students on elements of geometry and associated algebra. We conducted an experiment where students’ understanding of the definition of dot product of two vectors, cosine function and linear (in)dependence of vectors is evaluated, and address their mathematical activity to provide insight into the key elements of the problem they are solving. We use Bloom’s and SOLO taxonomy as a tool for the assessment of our findings. We obtained the data from written exams given to vector ...

Models of analog retrieval require a computationally cheap method of estimating similarity between a probe and the candidates in a large pool of memory items. The vector dot-product operation would be ideal for this purpose if it were possible to encode complex structures as vector representations in such a way that the superficial similarity of vector representations reflected underlying structural similarity. This paper describes how such an encoding is provided by Holographic Reduced Representations (HRRs), which are a method for encoding nested relational structures as fixed-width distributed representations. The conditions under which structural similarity is reflected in the dot-product rankings of HRRs are discussed.

Stragglers, Byzantine workers, and data privacy are the main bottlenecks in distributed cloud computing. Some prior works proposed coded computing strategies to jointly address all three challenges. They require either a large number of workers, a significant communication cost or a significant computational complexity to tolerate Byzantine workers. Much of the overhead in prior schemes comes from the fact that they tightly couple coding for all three problems into a single framework. In this paper, we propose Adaptive Verifiable Coded Computing (AVCC) framework that decouples the Byzantine node detection challenge from the straggler tolerance. AVCC leverages coded computing just for handling stragglers and privacy, and then uses an orthogonal approach that leverages verifiable computing to mitigate Byzantine workers. Furthermore, AVCC dynamically adapts its coding scheme to trade-off straggler tolerance with Byzantine protection. We evaluate AVCC on a compute-intensive distributed logistic regression application. Our experiments show that AVCC achieves up to 4.2× speedup and up to 5.1% accuracy improvement over the state-of-the-art Lagrange coded computing approach (LCC). AVCC also speeds up the conventional uncoded implementation of distributed logistic regression by up to 7.6×, and improves the test accuracy by up to 12.1%.

Cutting edge detectors push sensing technology by further improving spatial and temporal resolution, increasing detector area and volume, and generally reducing backgrounds and noise. This has led to a explosion of more and more data being generated in next-generation experiments. Therefore, the need for nearsensor, at the data source, processing with more powerful algorithms is becoming increasingly important to more efficiently capture the right experimental data, reduce downstream system complexity, and enable faster and lower-power feedback loops. In this paper, we discuss the motivations and potential applications for on-detector AI. Furthermore, the unique requirements of particle physics can uniquely drive the development of novel AI hardware and design tools. We describe existing modern work for particle physics in this area. Finally, we outline a number of areas of opportunity where we can advance machine learning techniques, codesign workflows, and future microelectronics technologies which will accelerate design, performance, and implementations for next generation experiments.

Let A be a commutative ring with nonzero identity, 1 ≤ n < be an integer, and R = A × A × • • • × A (n times). The total dot product graph of R is the (undirected) graph TD R with vertices R * = R\ 0 0 0 , and two distinct vertices x and y are adjacent if and only if x • y = 0 ∈ A (where x • y denote the normal dot product of x and y). Let Z R denote the set of all zero-divisors of R. Then the zero-divisor dot product graph of R is the induced subgraph ZD R of TD R with vertices Z R * = Z R \ 0 0 0. It follows that each edge (path) of the classical zero-divisor graph R is an edge (path) of ZD R. We observe that if n = 1, then TD R is a disconnected graph and ZD R is identical to the well-known zero-divisor graph of R in the sense of Beck-Anderson-Livingston, and hence it is connected. In this paper, we study both graphs TD R and ZD R. For a commutative ring A and n ≥ 3, we show that TD R (ZD R) is connected with diameter two (at most three) and with girth three. Among other things, for n ≥ 2, we show that ZD R is identical to the zero-divisor graph of R if and only if either n = 2 and A is an integral domain or R is ring-isomorphic to 2 × 2 × 2 .

Random graphs are increasingly becoming objects of interest for modeling networks in a wide range of applications. Latent position random graph models posit that each node is associated with a latent position vector, and that these vectors follow some geometric structure in the latent space. In this paper, we consider random dot product graphs, in which an edge is formed between two nodes with probability given by the inner product of their respective latent positions. We assume that the latent position vectors lie on an unknown one-dimensional curve and are coupled with a response covariate via a regression model. Using the geometry of the underlying latent position vectors, we propose a manifold learning and graph embedding technique to predict the response variable on out-of-sample nodes, and we establish convergence guarantees for these responses. Our theoretical results are supported by simulations and an application to Drosophila brain data.

This study focused on an investigation of Thai students’ comprehension of vectors using the model analysis technique, which is based on the matrix representation of quantum physics. This technique displays students’ knowledge states and the probability of applying each mental model to solve a given vector concept. A well-known vector concept test in physics education research named the Test of Understanding of Vectors (TUV) was translated into Thai language and validated by a group of physics lecturers. It was administered to 651 first-year engineering students at Prince of Songkla University via the online system at the beginning of the first semester. The three common mental states of vectors were identified as correct, popular incorrect, and other models. Model estimation of the model analysis revealed that although 54% of the students correctly answered vector addition, they still had difficulty with vector subtraction. Theoretically, both topics are considered as identical conc...

This paper presents multi-functional double-precision and quadruple-precision floating-point multiply-add fused (FPMAF) designs. The double-precision FPMAF design can execute adouble-precision floating-point multiply-add, or two single-precision floating-point multiplications, or a single-precision floating-point dot product. The quadruple-precision FPMAF can perform similar operations with quadruple, double and single precision operands. These architectures can perform a dot-product operation two times or more faster than a basic FPMAF design. The presented multi-functional designs are compared with basic double-precision and quadruple-precision FPMAF designs by ASIC syntheses. The syntheses results show that the proposed double-precision implementation has 8%more area than a standard double-precision FPMAF implementation, and the proposed quadruple-precision design has 12.5% more area than a standard quadruple-precision FPMAF. Both of the proposed designs have one more pipeline stage compared to the basic designs.

Data classification is one of the most fundamental tasks that can be accomplished by supervised machine learning. There exists a lot of algorithms, and they have the specific case of uses. Different classification methods follow different techniques to map the relationship between input and output. This article proposes an angle measurement-based classification technique called Support Directional Shifting Vectors (SDSV) to segment a spectral domain into regions with a very effective solution for classification problems. This method introduces two shifting vectors, named Support Direction Vector (SDV) and Support Origin Vector (SOV). These vectors are formed as a linear function to measure cosine-angle by the dot product of two separated data classes, named target data points and non-target data points. Considering the target class samples, the vectors get aligned in a way that the angle with the target class gets minimized, while the angle with the non-target class gets maximized. The error in the position of the linear function has been modeled as the loss function. Then, the vector position is updated iteratively by optimizing this loss function using a gradient descent algorithm. We have used this model to classify data from two different machine learning datasets to evaluate the performance of the proposed method. Finally, we have carefully examined the results and compared them with the other standard classification algorithms. In summary, the proposed SDSV algorithm has shown a comparable accuracy compared to different standard algorithms.

Machine learning models have been very popular nowadays for providing rigorous solutions to complicated real-life problems. There are three main domains named supervised, unsupervised, and reinforcement. Supervised learning mainly deals with regression and classification. There exist several types of classification algorithms, and these are based on various bases. The classification performance varies based on the dataset velocity and the algorithm selection. In this article, we have focused on developing a model of angular nature that performs supervised classification. Here, we have used two shifting vectors named Support Direction Vector (SDV) and Support Origin Vector (SOV) to form a linear function. These vectors form a linear function to measure cosine-angle with both the target class data and the non-target class data. Considering target data points, the linear function takes such a position that minimizes its angle with target class data and maximizes its angle with non-targ...

The in-memory computing paradigm with emerging memory devices has been recently shown to be a promising way to accelerate deep learning. Resistive processing unit (RPU) has been proposed to enable the vector-vector outer product in a crossbar array using a stochastic train of identical pulses to enable one-shot weight update, promising intense speed-up in matrix multiplication operations, which form the bulk of training neural networks. However, the performance of the system suffers if the device does not satisfy the condition of linear conductance change over around 1,000 conductance levels. This is a challenge for nanoscale memories. Recently, Charge Trap Flash (CTF) memory was shown to have a large number of levels before saturation, but variable non-linearity. In this paper, we explore the trade-off between the range of conductance change and linearity. We show, through simulations, that at an optimum choice of the range, our system performs nearly as well as the models trained using exact floating point operations, with less than 1% reduction in the performance. Our system reaches an accuracy of 97.9% on MNIST dataset, 89.1% and 70.5% accuracy on CIFAR-10 and CIFAR-100 datasets (using pre-extracted features). We also show its use in reinforcement learning, where it is used for value function approximation in Q-Learning, and learns to complete an episode the mountain car control problem in around 146 steps. Benchmarked to state-ofthe-art, the CTF based RPU shows best in class performance to enable software equivalent performance.

Sequential data naturally have different lengths in many domains, with some very long sequences. As an important modeling tool, neural attention should capture long-range interaction in such sequences. However, most existing neural attention models admit only short sequences, or they have to employ chunking or padding to enforce a constant input length. Here we propose a simple neural network building block called ChordMixer which can model the attention for long sequences with variable lengths. Each ChordMixer block consists of a positionwise rotation layer without learnable parameters and an element-wise MLP layer. Repeatedly applying such blocks forms an effective network backbone that mixes the input signals towards the learning targets. We have tested ChordMixer on the synthetic adding problem, long document classification, and DNA sequence-based taxonomy classification. The experiment results show that our method substantially outperforms other neural attention models. 1

Face recognition has been a fast growing, challenging and interesting area in real-time applications. A large number of face recognition algorithms have been developed from decades. Principal Component Analysis (PCA) [2][3] is one of the most successful techniques that has been used in face recognition. Four criteria for image pixel selection to create feature vector were analyzed: the first one has all the pixels considered by converting the image into gray plane, the second one is based on taking row mean in RGB plane of face image, the third one is based on taking column mean in RGB plane finally, the fourth criterion is based on taking row and column mean of face image in RGB plane and feature vector were generated to apply PCA technique. Experimental tests on the ORL Face Database [1] achieved 99.60% of recognition accuracy, with lower computational cost. To test the ruggedness of proposed techniques, they are tested on our own created face database where 80.60% of recognition accuracy is achieved. For a 128 x 128 image that means that one must compute a 16384 x 16384 matrix and calculate 16,384 eigenfaces. Computationally, this is not very efficient as most of those eigenfaces are not useful for our task. Using row mean and column mean reduces computations resulting in faster face recognition with nearly the same accuracy.

In a number of machine learning models, an input query is searched across the trained class vectors to find the closest feature class vector in cosine similarity metric. However, performing the cosine similarities between the vectors in Von-Neumann machines involves a large number of multiplications, Euclidean normalizations and division operations, thus incurring heavy hardware energy and latency overheads. Moreover, due to the memory wall problem that presents in the conventional architecture, frequent cosine similarity-based searches (CSSs) over the class vectors requires a lot of data movements, limiting the throughput and efficiency of the system. To overcome the aforementioned challenges, this paper introduces COSIME, a general in-memory associative memory (AM) engine based on the ferroelectric FET (FeFET) device for efficient CSS. By leveraging the one-transistor AND gate function of FeFET devices, current-based translinear analog circuit and winner-takeall (WTA) circuitry, COSIME can realize parallel in-memory CSS across all the entries in a memory block, and output the closest word to the input query in cosine similarity metric. Evaluation results at the array level suggest that the proposed COSIME design achieves 333× and 90.5× latency and energy improvements, respectively, and realizes better classification accuracy when compared with an AM design implementing approximated CSS. The proposed in-memory computing fabric is evaluated for an HDC problem, showcasing that COSIME can achieve on average 47.1× and 98.5× speedup and energy efficiency improvements compared with an GPU implementation.

We consider the problem of computing a matrixvector product Ax using a set of P parallel or distributed processing nodes prone to "straggling," i.e., unpredictable delays. Every processing node can access only a fraction s N of the Nlength vector x, and all processing nodes compute an equal number of dot products. We propose a novel error correcting code-that we call "Short-Dot"-that introduces redundant, shorter dot products such that only a subset of the nodes' outputs are sufficient to compute Ax. To address the problem of straggling in computing matrix-vector products, prior work uses replication or erasure coding to encode parts of the matrix A, but the length of the dot products computed at each processing node is still N. The key novelty in our work is that instead of computing the long dot products as required in the original matrix-vector product, we construct a larger number of redundant and short dot products that only require a fraction of x to be accessed during the computation. Short-Dot is thus useful in a communication constrained scenario as it allows for only a fraction of x to be accessed by each processing node. Further, we show that in the particular regime where the number of available processing nodes is greater than the total number of dot products, Short-Dot has lower expected computation time under straggling under an exponential model compared to existing strategies, e.g. replication, in a scaling sense. We also derive fundamental limits on the trade-off between the length of the dot products and the recovery threshold, i.e., the required number of processing nodes, showing that Short-Dot is near-optimal.

With ever increasing depth and width in deep neural networks to achieve state-of-the-art performance, deep learning computation has significantly grown, and dot-products remain dominant in overall computation time. Most prior works are built on conventional dot-product where weighted input summation is used to represent the neuron operation. However, another implementation of dot-product based on the notion of angles and magnitudes in the Euclidean space has attracted limited attention. This paper proposes DeepCAM, an inference accelerator built on two critical innovations to alleviate the computation time bottleneck of convolutional neural networks. The first innovation is an approximate dot-product built on computations in the Euclidean space that can replace addition and multiplication with simple bit-wise operations. The second innovation is a dynamic size content addressable memory-based (CAM-based) accelerator to perform bit-wise operations and accelerate the CNNs with a lower computation time. Our experiments on benchmark image recognition datasets demonstrate that DeepCAM is up to 523× and 3498× faster than Eyeriss and traditional CPUs like Intel Skylake, respectively. Furthermore, the energy consumed by our DeepCAM approach is 2.16× to 109× less compared to Eyeriss.

Data classification is one of the most fundamental tasks that can be accomplished by supervised machine learning. There exists a lot of algorithms, and they have the specific case of uses. Different classification methods follow different techniques to map the relationship between input and output. This article proposes an angle measurement-based classification technique called Support Directional Shifting Vectors (SDSV) to segment a spectral domain into regions with a very effective solution for classification problems. This method introduces two shifting vectors, named Support Direction Vector (SDV) and Support Origin Vector (SOV). These vectors are formed as a linear function to measure cosine-angle by the dot product of two separated data classes, named target data points and non-target data points. Considering the target class samples, the vectors get aligned in a way that the angle with the target class gets minimized, while the angle with the non-target class gets maximized. The error in the position of the linear function has been modeled as the loss function. Then, the vector position is updated iteratively by optimizing this loss function using a gradient descent algorithm. We have used this model to classify data from two different machine learning datasets to evaluate the performance of the proposed method. Finally, we have carefully examined the results and compared them with the other standard classification algorithms. In summary, the proposed SDSV algorithm has shown a comparable accuracy compared to different standard algorithms.

This work presents an energy-efficient SRAM with embedded dot-product computation capability, for binary-weight convolutional neural networks. A 10T bit-cell based SRAM array is used to store the 1-b filter weights. The array implements dotproduct as a weighted average of the bit-line voltages, which are proportional to the digital input values. Local integrating ADCs compute the digital convolution outputs, corresponding to each filter. We have successfully demonstrated functionality (> 98% accuracy) with the 10,000 test images in the MNIST handwritten digit recognition dataset, using 6-b inputs/outputs. Compared to conventional full-digital implementations using small bit-widths, we achieve similar or better energy-efficiency, by reducing data transfer, due to the highly parallel in-memory analog computations.

Data classification is one of the most fundamental tasks that can be accomplished by supervised machine learning. There exists a lot of algorithms, and they have the specific case of uses. Different classification methods follow different techniques to map the relationship between input and output. This article proposes an angle measurement-based classification technique called Support Directional Shifting Vectors (SDSV) to segment a spectral domain into regions with a very effective solution for classification problems. This method introduces two shifting vectors, named Support Direction Vector (SDV) and Support Origin Vector (SOV). These vectors are formed as a linear function to measure cosine-angle by the dot product of two separated data classes, named target data points and non-target data points. Considering the target class samples, the vectors get aligned in a way that the angle with the target class gets minimized, while the angle with the non-target class gets maximized. The error in the position of the linear function has been modeled as the loss function. Then, the vector position is updated iteratively by optimizing this loss function using a gradient descent algorithm. We have used this model to classify data from two different machine learning datasets to evaluate the performance of the proposed method. Finally, we have carefully examined the results and compared them with the other standard classification algorithms. In summary, the proposed SDSV algorithm has shown a comparable accuracy compared to different standard algorithms.

In-memory computing' is being widely explored as a novel computing paradigm to mitigate the well known memory bottleneck. This emerging paradigm aims at embedding some aspects of computations inside the memory array, thereby avoiding frequent and expensive movement of data between the compute unit and the storage memory. In-memory computing with respect to Silicon memories has been widely explored on various memory bit-cells. Embedding computation inside the 6 transistor (6T) SRAM array is of special interest since it is the most widely used on-chip memory. In this paper, we present a novel in-memory multiplication followed by accumulation operation capable of performing parallel dot products within 6T SRAM without any changes to the standard bitcell. We, further, study the effect of circuit non-idealities and process variations on the accuracy of the LeNet-5 and VGG neural network architectures against the MNIST and CIFAR-10 datasets, respectively. The proposed in-memory dot-product mechanism achieves 88.8% and 99% accuracy for the CIFAR-10 and MNIST, respectively. Compared to the standard von Neumann system, the proposed system is 6.24× better in energy consumption and 9.42× better in delay.

Transformers have emerged as a powerful tool for a broad range of natural language processing tasks. A key component that drives the impressive performance of Transformers is the self-attention mechanism that encodes the influence or dependence of other tokens on each specific token. While beneficial, the quadratic complexity of self-attention on the input sequence length has limited its application to longer sequences - a topic being actively studied in the community. To address this limitation, we propose Nyströmformer - a model that exhibits favorable scalability as a function of sequence length. Our idea is based on adapting the Nyström method to approximate standard self-attention with O(n) complexity. The scalability of Nyströmformer enables application to longer sequences with thousands of tokens. We perform evaluations on multiple downstream tasks on the GLUE benchmark and IMDB reviews with standard sequence length, and find that our Nyströmformer performs comparably, or in ...

Aims/ Objectives: Given research is an attempt to establish a connection between various groups of numbers The author aims to justify the statement: is true as an example of the so-called “Ternary Groups” Definition of “ Ternary” very much depends on vector properties and it’s components Keywords: Three Component Vector ,Bayesian Decomposition ,Matrix of Rotation 1 Introduction Let’s first write the definition of a three component vector : It is a vector the components of which are orthonormal and is written as The indice x,y,z are commonly referred to as i,j,k units along x,y,z axis The question is whether such reference is correct As an example we use most commonly used dot and cross product often referred to as scalar and vector multiplication The outcomes of these two products result in different classes of number presentations i.e., a scalar and a vector respectively Having said that we want to establish a link between numeric values and their vector representation Figure 1

When we administered the Test of Understanding of Vectors (TUV) to students who were completing a physics university remedial course (which covers subjects of a traditional high school physics course), we observed that they had considerable difficulties in interpreting the dot product as a projection. As a result of this, we decided to design a tutorial worksheet to guide students through the development of this skill. The worksheet was implemented with 288 students of the same course in another semester. Upon using the TUV to evaluate the students' understanding, we confirmed that the tutorial worksheet facilitated their learning. This tutorial worksheet is presented in the Appendix and might be used by other physics instructors teaching this material in high schools, colleges or universities.