
In this post, I explain the concepts of LCC (linear constant-coefficient) systems , impulses, impulse responses and LCCDEs (Linear Constant-Coefficient Difference Equations), working toward an intuitive understanding of convolution.

This isn't a comprehensive theoretical treatment — rather, it's a practical introduction aimed at building intuition through grounded examples and relatable analogies.

# Igor the Robot

Let’s say a robot (we will call him Igor) reports its position $x[k]$ every $T=0.1s$. That is a sampling rate of 10Hz.

Suppose we want to estimate Igo’s velocity using only its position data $x[k]$. 

A simple way to do this is to approximate the derivative using a simple backward difference (fancy term for derivative):

 

$
y[k] = \frac{x[k] - x[k-1]}{T}
$

This is our discrete-time system: it takes two samples and computes a scaled difference — essentially, a numerical derivative.

Before talking about impulse response, convolution (in a latter post), there is 2 proprieties worth defining, mainly, linearity and Time in-variance. 

### Linearity and Time-Invariance

Our system is Linear and Time Invariant **(LTI)** because: 

1. If you double the input, the output doubles (linearity)
2. Delaying the inputs, delays the output by the same amount (time in-variance)

Together, this makes it an **LTI system** (Linear Time-Invariant), and LTI systems have a powerful property:

> They can be fully characterized by their response to a single impulse response.
> 

But, before talking about impulse response, Its worth looking at what a normal system response looks like.  

### A system’s response

From our velocity estimator, 

$$
y[k] = \frac{x[k] - x[k-1]}{T} = \frac{x[k] - x[k-1]}{0.1} = 10x[k] -10x[k-1]\\ 
$$

This tells us how the system responds to any arbitrary input signal $x[k]$ Notice the constant coefficients). 

Suppose we read the  following data for what the robot moved. Remember x[k] is the position we read.

$$
x = [0, 0.5,1.1,1.6,2.0]
$$

| k | $x[k] (m)$ | $x[k−1] (m)$ | $y[k] (m/s)$ |
| --- | --- | --- | --- |
| 0 | 0.0 | — | undefined or 0 |
| 1 | 0.5 | 0.0 | 10(0.5) - 10(0.0) = 10 |
| 2 | 1.1 | 0.5 | 10(1.1) - 10(0.5) = 6 |
| 3 | 1.6 | 1.1 | 5 |
| 4 | 2.0 | 1.6 | 4 |

So, the output sequence is:

$$
y[k]=\set{0, 5,6,5,4,...} (m/s)
$$

This gives the instantaneous velocity estimate between samples — a discrete approximation of Igor's speed.

# What’s an Impulse?

Imagine you tap a system very briefly and then watch how it reacts.

- Like striking a bell once (input) and listening to how it rings (response).
- Like snapping your fingers (input) and measuring the sound that follows (response).
- In physics, a hammer hit (impulse) causes vibration (response).
- In electronics, a voltage spike shows how a circuit handles sudden changes.

Mathematically we represent the impulse as as: 

$$
x[k] = \delta[k] = 
\begin{cases}
1, & k =0 \\
0, & \text{otherwise}
\end{cases}
$$

This is the simplest possible input.

# Impulse response

Now, let’s feed this impulse into our system and observe the output.

Given 

$$
y[k] = 10x[k] - 10x[k-1]
$$

Let $x[k] = \delta[k]$

| k | Position `$x[k]$`  | Previous Position $x[k−1]$ | $y[k] =10x[k] - 10x[k]$ |
| --- | --- | --- | --- |
| -1 | — | — | 0 |
| 0 | 1 | 0 | $(1 - 0)/0.1 = 10$ |
| 1 | 0 | 1 | $(0 - 1)/0.1 = -10$ |
| 2+ | 0 | 0 | 0 |

So the output (i.e. impulse response) is 

Thus, the **impulse response** is:

$$
h[k] = \set{\frac{1}{T}, -\frac{1}{T}, 0} =  \set{10,-10}
$$

or more generally:

$$
h[k] = \frac{1}{T}\delta[k] - \frac{1}{T}\delta[k-1] = 10\delta[k] - 10\delta[k-1]
$$

In essence, an impulse response $h[k]$ is the response $y[k]$ when $x[k] = \delta[k]$.

### Why care?

The impulse response is an important idea in Time Domain Analysis is that: 

> If you know how a system responds to an impulse, you can predict its output to **any** signal by using **convolution**
> 

$$
y[k] = x[k] * h[k] 
$$

Here’s how the system might be implemented in code:

```matlab
float h[2] = {10.0, -10.0}; // coefficients
float y = h[0]*x[k] + h[1]*x[k-1];

```

### LCCDEs

A Linear Constant Coefficient Difference Equation is an **equation** that **describes** a discrete-time an LTI system. This equation models how a system behaves over time. That is how current inputs and past ouputs/input determine the present output. LCCDEs are particular useful to us when it comes to actual implementation of systems like filters. 

For example, our velocity estimator is characterized by the equation

$$
y[k] = x[k] - x[k-1]
$$

This is the discrete equivalent of a derivative. More generally, LCCDEs serve the same role in DSP as differential equations do in continuous systems.

Solving an LCCDE with the impulse input $x[k] = \delta[k]$ yields the **impulse response** h[k]h[k]h[k]. Once you have $h[k]$, you can compute the output for *any* input using convolution:

$$
⁍
$$

This is the foundation of **time-domain DSP**.

# Connecting All Together

Let’s connect the dots:

The **impulse** $\delta[k]$ probes how the system behaves.

The system’s response, $h[k]$, **characterizes it fully**.

To compute the output for **any** input $x[k]$, we convolve it with $y[k]=x[k]∗h[k]$

Essentially, once you know $h[k]$, you can fully simulate, analyze, and implement the system.

An intuitive guide to how linear time-invariant systems respond to impulses, with practical examples from signal processing.

Understanding Impulse Responses in Discrete-Time Systems

In ancient Greece, by the sea, the followers of Pythagoras met in secret. They gathered in the dark, lit by flickering lamps, bound by a shared belief that numbers were sacred. Numbers, they thought, were the truth of the universe, the language of the gods. Above their school’s entrance, the words read: **"All is number."**

Among them was Hippasus, a young mathematician devoted to his craft. He, like the others, believed only in rational numbers. Mainly, every number can be written as a simple ratio of $\frac{a}{b}$. For example, 2/3 or 4/1, etc. Numbers were whole, clean, and perfect. But one night, Hippasus found something that broke this perfect world.

He discovered that some numbers, like the square root of 2, could not be expressed as fractions. No matter how he tried, they did not fit. He asked his teacher why would that be. The answer eluded his mentor as well, leaving him in the dark.  In a moment of profound despair, the teacher drowned his student.

### What Makes a Simple Number So Troubling?

Although the murder of Hippasus is  probably a matter of legend the secret was real, and certainly dangerous enough to the beliefs of those who knew about it—a warning whispered among those who knew. Why?

Imagine a right angle triangle with 2 sides equal to one. From Pythagoras’s theorem :

![](/images/blog/pythagorean.png)


We know that the hypotenuse's length squared is $1^2+1^2=2$, making the length of the hypotenuse itself $\sqrt2$ . The number which when multiplied by itself makes $2$. But what is that number? The square root of 2 isn't 1 because $1 \times 1 = 1$.  It’s not $2$  because $2 \times 2 = 4$. It's something in between but what is it?

At first, this wasn't a problem for the Pythagoreans. It was obviously a ratio of two whole numbers. They only had to figure out what that ratio was. After more and more frantic attempts, a horrible discovery was made. There is NO ratio that will produce $\sqrt{2}$. —it simply cannot be done. It's what we now call an irrational number. It can't be represented as a ratio of whole numbers.

This was a number that defied all their rules—a number we now call irrational. It can’t be expressed as a ratio of whole numbers, no matter how hard you try. For the Pythagoreans, this was not just a mathematical anomaly—it was a threat. They lived by the belief that numbers were the foundation of the universe, and every number fit into a neat, orderly system. But irrational numbers were a dark secret, a crack in their perfect world of order. To acknowledge these numbers was to admit that their entire understanding was flawed. Worse yet, it was to admit that the universe itself was not as perfect and orderly as they had believed.

Hippasus’s discovery was more than a flaw in their math; it was a challenge to their core beliefs. It was proof that the world was not built entirely on whole numbers and ratios. For the Pythagoreans, that was a truth too dangerous to accept. One that may have cost Hippasus, his life. 

### Troubles of Belief : Facing Our Fallibility

The case of Hippasus is far from unique. Throughout history, individuals—be they scientists, philosophers, or prophets—have encountered ideas that threatened the established belief systems of their time. These ideas were often met with ridicule, hostility, and at times, even violence.

In the “Myth of the framework”, Karl Popper makes an important remark that resonates with me. That is, 

> The history of intellectual progress is a history of prejudice and dogma, held with tenacity, often accompanied by intolerance and fanaticism. Yet, even amidst religious wars and persecutions, there is a brighter side. It is heartening to remember that countless individuals, from ancient to modern times, have been willing to live and die for their convictions—for ideas they believed to be true. Man, we may say, is not so much a rational animal as an ideological animal.
> 

“As humans, we are ready to die for our ideas!”

This noble tendency to cling to our beliefs is a double-edged sword. On one hand, it is honorable to defend and even die for one’s convictions. But on the other, what a waste, given the high likelihood that you, me and everyone is wrong. Is it worth it to die for a bad idea? What about living? 

Considering how little we know—both individually and collectively—it is crucial to acknowledge our limitation. The fallibility of human knowledge is an undeniable truth, one that we must confront if we are to engage with reality. 

This is the basis of my thesis : **As Human, we are fallible!** 

In the following paragraphs, I will explore the implications of this recognition—the necessity of being open to ideas that challenge our beliefs, and the importance of a method for determining when and how to abandon those beliefs.—Too ambitious? Probably!

I should perhaps clarify: when I refer to scientific theories, mathematics, or philosophy—fields I intend to borrow from—it is not to champion the conclusions of modern science or present it as an ultimate authority. I am not here to evangelize for science. My focus is on addressing the problems within my religious beliefs. I am interested in the methods and principles of these disciplines because they offer a framework for understanding how we arrive at knowledge—and, crucially, when and how we should be ready to let go of our beliefs when confronted with new evidence. 

Since I’m abandoning my beliefs, I find it fit to explore such a theory.

An essay about humanity's fallibility and the need for openness to ideas that challenge deeply held beliefs.

Dangerous Idea


### The AGI hype train has hit some heavy traffic.

While futurists and fundraisers used to make bullish predictions about artificial general intelligence, they’ve become quieter lately. Peter Thiel — the tech billionaire and rumored vampire — says Silicon Valley big brains have lost enthusiasm for AGI.

“Elon’s not talking about it anymore and Larry Page is off to Fiji and doesn’t seem to be working on it quite as hard,” Thiel said at a recent event.

While futurists and fundraisers used to make bullish predictions about artificial general intelligence, they’ve become quieter lately. Peter Thiel — the tech billionaire and rumored vampire — says Silicon Valley big brains have lost enthusiasm for AGI.

### IT is healing
Can self-healing security software fight cybercrime?

Thiel described Musk as “a weathervane for the zeitgeist,” who’s stopped talking about AGI because interest has declined.

Scientists are also increasingly skeptical. A recent study paper posited that AGI is “in principle impossible,” while other researchers have mocked the term’s proponents.

“I have yet to come across work on AGI that I can take seriously,” tweeted Abeba Birhane, a cognitive scientist based at University College Dublin.

### The path towards AGI increasingly appears — at best — a long one.

Imbuing machines with human-like intelligence remains immensely challenging. As Melanie Mitchell, a computer science professor at Portland State University noted in a preprint paper published last year on arXiv:


>Since its beginning in the 1950s, the field of artificial intelligence has cycled several times between periods of optimistic predictions and massive investment
<cite>Alexender Toto</cite>

Critics fear that another AI winter is coming. Hyping AGI helped spurred enormous investment in artificial intelligence, but overconfident predictions could be detrimental to progress in the field — if the real-world advances prove disappointing.

While futurists and fundraisers used to make bullish predictions about artificial general intelligence, they’ve become quieter lately. Peter Thiel — the tech billionaire and rumored vampire — says Silicon Valley big brains have lost enthusiasm for AGI.

The AGI hype train is running out of steam


Lithium-ion batteries have played a vital role in the development of electric vehicles and we love them for that. But at the same time, lithium is expensive to produce, unstable in high temperatures, and a finite resource whose mining often comes with supply chain problems.

Battery researchers and manufacturers have been assiduously searching for a more sustainable replacement. One intriguing alternative they’ve discovered is another chemical element: sodium.


### The reason some bootcamp grads are set up to fail
* In my experience, not all bootcamps are created equal. They vary widely in acceptance processes, curriculum, program structure, and quality of instructors. They’re designed to push candidates through courses that will have them writing some code fairly quickly.

* As a result, programs are forced to strip away a lot of fundamentals — those basics that help developers understand the “why” behind the code they’re writing.

* When young software developers learn by copying and pasting, it can make troubleshooting difficult when they come across something that doesn’t fit the pattern they’re used to.

* Small businesses and startups are those most likely to hire developers without degrees, but I’ve seen too many bootcamp grads take jobs at startups only to find they haven’t learned enough to make any real impact.

### 1. Look for bootcamp grads with personal projects
The best bootcamp grads are those who already considered themselves hobbyists. They didn’t enter a program expecting to learn everything they needed to know in 12 weeks.

They were already passionate about software development and enrolled in a bootcamp for some structure and guidance, and to level up their skills.

### 2. Adjust your interview process to test for fundamentals
Because bootcamps are, by nature, time-constrained programs, some will blow right past the heavy lifting of helping developers understand why things work the way they do.

The result is that some graduates come out of bootcamps using pattern recognition as their primary skill and an understanding of how to get by with copying and pasting code.

### 3. Hire natural problem solvers and nurture their intellectual curiosity
All engineering is problem-solving. Junior developers won’t be architecting anything right away, but you should look for people who show a predilection for understanding the problem they’re trying to solve and evaluating potential paths forward.

Even when your newest developers are working as order takers, they should be thinking critically about why those orders are issued. A good mentor can help initiate those conversations about the “why” behind each assignment and help new hires see what they might be missing.

### 4. Set new hires up for success with appropriate oversight
If you’re hiring someone fresh out of bootcamp, don’t expect them to come in and start developing complex applications by themselves. Most bootcamp grads have never worked in a real-world programming environment before, and you can’t expect them to be self-directed from day one.

You need a strong new-hire onboarding process and support system that makes new hires feel comfortable asking questions and continuing to learn.

Lithium-ion batteries have played a vital role in the development of electric vehicles and we love them for that. But at the same time, lithium is expensive to produce, unstable in high temperatures, and a finite resource whose mining often comes with supply chain problems.