Waddah Moghram, Ph.D.

This week's Tuesday Tutorial takes our exploration of neural networks to the next level by applying them to Universal Differential Equations: https://lnkd.in/gbyZfBUU. In this installment, we generate data and train a neural net to partially learn a Lotka-Voltera system using Collimator. #controlsystems #controlsengineering #controltheory #smartsystems #engineering #python #JAX #collimator #simulation #technology #machinelearning #ml #ai

22

𝗜𝗻 𝘁𝗵𝗲 𝗰𝗼𝗻𝘁𝗲𝘅𝘁 𝗼𝗳 𝗱𝗲𝗿𝗶𝘃𝗶𝗻𝗴 𝘁𝗵𝗲 𝗹𝗶𝗸𝗲𝗹𝗶𝗵𝗼𝗼𝗱 𝗿𝗮𝘁𝗶𝗼 𝘁𝗲𝘀𝘁 𝗳𝗼𝗿 𝗮𝗻𝗼𝗺𝗮𝗹𝘆 𝗱𝗲𝘁𝗲𝗰𝘁𝗶𝗼𝗻, which involves detecting deviations from an expected signal behavior, the typical approach focuses on the null hypothesis H0: x[n] = w[n]. Here, w[n] represents the usual distribution of the received signal or noise. In classical statistics, this problem is addressed through a significance test. However, an alternative approach yields an intriguing outcome: the decision of the detector relies on the estimated mutual information, and naturally incorporates operators like the Kullback-Leibler divergence (KLD) as a solution. Essentially, the presence of an anomaly generates new information. This concept involves utilizing the estimated quantized cumulative distribution function (CDF) under H0 to produce uniformly distributed random variables (r.v.s) and measure the divergence from uniformity using KLD. Please refer to the image below for the specific test. If you require additional information, feel free to reach out to me. If you're interested in joining a short masterclass to delve deeper into this algorithm, where we'll derive it and examine an example code demonstrating its functionality, please let me know either via DM or in the comments. PS this post is devoted to my KLD friend, Daniel Aronovich #AnomalyDetection #SignalProcessing #DataAnalytics #Statistics #MachineLearning #DataScience #InformationTheory #KullbackLeibler #MutualInformation #LikelihoodRatioTest #QuantizedCDF

44

10 Comments

A 13-pager Tutorial about Linear-Regression just added to the circuit of knowledge section in my website, making it the 30'th tutorial I wrote since I opened this website 2 months ago. I PDF'ed it and attached here. Enjoy! For more tutorials: https://lnkd.in/g8RB2pQf #linearregression #machinelearning #datascience #LeastSquares #AI #RadarLove

23

1 Comment

🔗 Simulate PyTorch Models with Simulink! 🚀 Engineers, unlock new possibilities by simulating PyTorch and other Python-based models directly in Simulink using co-execution blocks. This integration streamlines your workflow, enabling seamless model testing and validation. 🔍 What's Inside: >> Co-execution of Python models in Simulink << >> Enhanced simulation and validation processes << 👉 Dive into the details: https://spr.ly/6041o1cOX #DeepLearning #Simulink #Python #ModelSimulation #Engineering #MATLAB #MAthWorks #PyTorch

9

Today, I came across a short video from Waymo giving a high-level overview of their tech. It’s clearly not aimed at tech nerds like me, but it sparked some thoughts about deploying robots in the real world. One point mentioned was "capturing the long tail." But can we really capture it? If we could, it wouldn’t be a long tail, right? Humans can’t predict every scenario (thank God!), but we can detect and filter out glitches, like the black cat in The Matrix. We reckon the existence of the glitch, work around it, and keep going, instead of hitting a truck thinking it was a bridge seen from a distance because we were biased by lots of bridges in our training dataset (yup, Tesla famous accident). Another interesting thing is the massive number of sensors Waymo’s cars use. At first, it seems excessive, and probably crazily expensive, but maybe it’s a clever play to shape regulations and build a solid protective moat 🤔 https://lnkd.in/greVa4am

1

3 𝙈𝙞𝙣𝙪𝙩𝙚𝙨 𝙤𝙣 𝙉𝙤𝙣𝙡𝙞𝙣𝙚𝙖𝙧 𝙇𝙚𝙖𝙨𝙩-𝙎𝙦𝙪𝙖𝙧𝙚𝙨 - 𝙏𝙝𝙚 𝙂𝙖𝙪𝙨𝙨 𝙉𝙚𝙬𝙩𝙤𝙣 𝘼𝙡𝙜𝙤𝙧𝙞𝙩𝙝𝙢 Hey! I just recorded a video explaining nonlinear least squares estimation. It's a fast converging solution that's often used in industry. Check it out here: #ML #AI #optimization #NLS #GaussNewton 

7

👉 Learn how to simulate pre-trained models from #PyTorch and #TensorFlow directly in #Simulink with the new co-execution Simulink blocks. https://spr.ly/6041oRNPc

3

🤖 Check out the MATLAB Deep Learning Hub on GitHub! What you'll find: 🧠 Examples and tutorials for deep learning concepts 🚀 Pretrained models to kickstart your project Get Started: 🖱️ Go to the MATLAB Deep Learning Hub on GitHub. 👀 Browse the resources. ⭐ Download, star, or fork your favorites and apply them to your projects! Click the link in our bio! #DeepLearning #MachineLearning #AI #MATLAB #GitHub #LearnToCode

1

"Are large language models superhuman chemists?" So what these researchers did was make a test -- a benchmark. They made a test of 7,059 chemistry questions, spanning the gamut of chemistry: computational chemistry, physical chemistry, materials science, macromolecular chemistry, electrochemistry, organic chemistry, general chemistry, analytical chemistry, chemical safety, and toxicology. They recruited 41 chemistry experts to carefully validate their test. They devised the test such that it could be evaluated in a completely automated manner. This meant relying on multiple-choice questions rather than open-ended questions more than they wanted to. The test has 6,202 multiple-choice questions and 857 open-ended questions (88% multiple-choice). The open-ended questions had to have parsers written to find numerical answers in the output in order to test them in an automated manner. In addition, they ask the models to say how confident they are in their answers. Before I tell you the ranking, the researchers write: "On the one hand, our findings underline the impressive capabilities of LLMs in the chemical sciences: Leading models outperform domain experts in specific chemistry questions on many topics. On the other hand, there are still striking limitations. For very relevant topics the answers models provide are wrong. On top of that, many models are not able to reliably estimate their own limitations. Yet, the success of the models in our evaluations perhaps also reveals more about the limitations of the exams we use to evaluate models -- and chemistry -- than about the models themselves. For instance, while models perform well on many textbook questions, they struggle with questions that require some more reasoning. Given that the models outperformed the average human in our study, we need to rethink how we teach and examine chemistry. Critical reasoning is increasingly essential, and rote solving of problems or memorization of facts is a domain in which LLMs will continue to outperform humans." "Our findings also highlight the nuanced trade-off between breadth and depth of evaluation frameworks. The analysis of model performance on different topics shows that models' performance varies widely across the subfields they are tested on. However, even within a topic, the performance of models can vary widely depending on the type of question and the reasoning required to answer it." And with that, I'll tell you the rankings. You can log in to their website at ChemBench.org and see the leaderboard any time for the latest rankings. At this moment I am seeing: gpt-4: 0.48 claude2: 0.29 GPT-3.5-Turbo: 0.26 gemini-pro: 0.25 mistral_8x7b: 0.24 text-davinci-003: 0.18 Perplexity 7B Chat: 0.18 galactica_120b: 0.15 Perplexity 7B online: 0.1 fb-llama-70b-chat: 0.05

3

When John Hopfield and Geoffrey Hinton won the Nobel Prize in Physics and their works were related to AI, I was curious about how these two subjects were correlated. Fundamentally from a physical aspect working of brains and computers have one thing in common: energy, the thermodynamics fundamental of the Efficient Compute Frontier. The brain consists of many neurons connected forming synapses. When we see or sense something, neurons excite and generate electric pulses of ions called Action potential. This flows from one neuron to another through synapse, by neurotransmitter. All we can consider is an ion transfer in and out of neurons, which I described little in my previous post. Now learning, a repetitive action, a repetitive excitation to the neuron generally called Long-term potentiation strengthens the synaptic transmissions. When this happens in the hippocampus region of the brain it's our cognitive learning. In a computer, there are CPU, RAM, ROM, and IO devices. CPU is the processing part. RAM and ROM is memory. Billions of interconnected transistors made logic gates and millions of gated formed CPUs. DRAM is made of MOSFETs and capacitors storing bits. In ROM it's a floating gate MOSFET storing one bit. When a computer processes data, a lot of digital signals flow from transistor to transistor. Similar to the brain's Action potential flows from neuron to neuron. The neurotransmitters in strengthen(trained) synapses act as weights, controlling impulses from node to node. It can be + (Excitory) or - ( Inhibitory) and these billions of depth neural networks have weights trained from this Long-term potentiation. Now when we compare between brain and the computer we can see first the information density. When we see one picture, our brains store that learning weight in neurons in the analog form of ion receptors. This means one single neuron can hold the weight value from multiple training. But in computers, we need massive memory for training where to store a single photo we need thousands of transistors. The second thing is the power to learn something the amounts of ion exchange in neurons needed way less power. Compared to the computer where a 2+2 = 4 process takes significant power. Storing a single bit in EEPROM needs so power to inject the hot carrier. Bitwise it's too less efficient in digital computers no matter how much computer capability we have. The amount of energy needed to process in this MOSFET-based CPU, and GPUs is still too high compared to the brain which is nothing like a super-efficient Analog computer. Mythic Has already developed analog computers, compute in memory solutions that are faster than conventional CPUs and GPUs. Rather than high TDP processors the problem statement is getting solved in a different way. I believe analog will be a computation feature where machines will learn more in the continuous time domain rather than discrete. #science #nature #electronics #engineering #biology #electronics

29

2 Comments

🚀 AI with MATLAB: Reflecting on 2024’s Highlights Check out this blog post to explore 3 major milestones for AI with MATLAB this year: 👉 LLMs with MATLAB: A new repository was created for you to access and interact with LLMs using MATLAB. 👉 Enhanced Integration with Python: You can now import PyTorch 2.0 models and simulate Python-based AI models in Simulink. 👉 Verification and Validation for AI: MATLAB introduced tools for explainability and constrained deep learning to build confidence in AI, especially for safety-critical applications. https://spr.ly/6045vL2DX #AI #MATLAB #MachineLearning #DeepLearning

2

👉 Join us on October 31st to discover how AI is transforming Partial Differential Equations (PDEs) with Physics-Informed Neural Networks (PINNs) and more! Join our webinar to explore the fusion of AI, physics, and math, with hands-on MATLAB demos for your research. #AI #PDEs #MachineLearning #MATLAB https://spr.ly/6043UxY47

2

🚨Paper Alert 🚨 ➡️Paper Title: Neural Circuit Architectural Priors for Quadruped Locomotion 🌟Few pointers from the paper 🎯Learning-based approaches to quadruped locomotion commonly adopt generic policy architectures like fully connected MLPs. As such architectures contain few inductive biases, it is common in practice to incorporate priors in the form of rewards, training curricula, imitation data, or trajectory generators. 🎯 In nature, animals are born with priors in the form of their nervous system's architecture, which has been shaped by evolution to confer innate ability and efficient learning. 🎯For instance, a horse can walk within hours of birth and can quickly improve with practice. Such architectural priors can also be useful in ANN architectures for AI. 🎯In this work, Authors have explored the advantages of a biologically inspired ANN architecture for quadruped locomotion based on neural circuits in the limbs and spinal cord of mammals. 🎯Their architecture achieves good initial performance and comparable final performance to MLPs, while using less data and orders of magnitude fewer parameters. 🎯Their architecture also exhibits better generalization to task variations, even admitting deployment on a physical robot without standard sim-to-real methods. This work shows that neural circuits can provide valuable architectural priors for locomotion and encourages future work in other sensorimotor skills. 🏢Organization: New York University 🧙Paper Authors:  Nikhil Bhattasali, Venkatesh Pattabiraman, Lerrel Pinto, Grace Lindsay 1️⃣Read the Full Paper here: https://lnkd.in/gb9hWsVn 2️⃣Project Page: https://lnkd.in/gaYcHs7R 🎥 Be sure to watch the attached Demo Video - Sound on 🔊🔊 Find this Valuable 💎 ? ♻️REPOST and teach your network something new Follow me 👣, Naveen Manwani, for the latest updates on Tech and AI-related news, insightful research papers, and exciting announcements.

12

For developers and anyone working on web applications, understanding how to send GET requests using cURL is essential. cURL, a command-line tool, allows users to transfer data from or to a server with ease. This post explores the fundamental steps needed to initiate GET requests, detailing the syntax and possible parameters that can be utilized. By mastering these principles, developers can efficiently interact with APIs and retrieve necessary data for their applications. More information can be found at

1