LinuxInternals.org

by Joel Fernandes

Understanding Critical Section Behavior With Ftrace Output

| Comments

The ftrace irqsoff, preempt and preemptirqsoff tracers are amazing tools identify paths in Linux kernel code that block tasks from getting CPU-time. Essentially during these paths, the scheduler has its hands tied behind its back. There are mainly 2 cases where this can happen.

Local Interrupts disabled (masked): All CPU cores have a bit in a ‘flags’ or status register, that is used to mask interrupts on that particular CPU. For x86 architectures, its in a FLAGS register called the IF bit. With this bit set, timer interrupts wont be received without which the scheduler wont get a chance to run and schedule something else.

Preemption disabled: Preemption is often disabled in the kernel when acquiring spinlocks. We went over the mechanics of spinlocks previously. There are other cases where preepmtion may be disabled as well. With preemption turned off, the scheduler will not preempt an existing task and prevent giving some other task a chance to run.

Both these cases introduce latency to runnable tasks and so they are bad for predictablity and real-timeness. Disabling interrupts is worse than turning off preemption because this also has the added effect of introducing latency to other interrupts.

The tracers I just mentioned are called latency tracers and they trace the total latency and optionally the functions that are executed between when irqs are turned off and on, or when preempt is turned off and on. They are documented well so I wont go over them here.

One thing I want to mention is about the preemptirqsoff tracer, it starts when when either preempt or irqs are turned off, and stops tracing only when both preemption and irqs are turned back on. This makes the preemptirqsoff tracer excellent in tracing kernel code paths that are blocking a task because regardless of whether preemption or irqs causing the latency, it treats both these cases identical or interchangeable. Borrowing from the amazing documentation, the following example illustrates this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
Knowing the locations that have interrupts disabled or
preemption disabled for the longest times is helpful. But
sometimes we would like to know when either preemption and/or
interrupts are disabled.

Consider the following code:

    local_irq_disable();
    call_function_with_irqs_off();
    preempt_disable();
    call_function_with_irqs_and_preemption_off();
    local_irq_enable();
    call_function_with_preemption_off();
    preempt_enable();

The irqsoff tracer will record the total length of
call_function_with_irqs_off() and
call_function_with_irqs_and_preemption_off().

The preemptoff tracer will record the total length of
call_function_with_irqs_and_preemption_off() and
call_function_with_preemption_off().

But neither will trace the time that interrupts and/or
preemption is disabled. This total time is the time that we can
not schedule. To record this time, use the preemptirqsoff
tracer.

It appears though that function tracing in preempt tracers is a bit broken and I posted a fix for a bug which prevent tracing functions which were called while interrupts were enabled and preemption is disabled. So in the above example, call_function_with_preemption_off wouldn’t be traced. It still tracks the max latency and reports it, but it would omit the tracing of this function.

Coming back to the original topic of understanding critical section kernel behavior, the beautiful ftrace output for latency tracers show you a set of flags on the left beside every function traced which illustrates what is going on with preemption and interrupts in the traced function. Here I will go over a set of trace outputs and show you some interesting things.

Tying 2 Voltage Sources/signals Together

| Comments

Recently I asked a question on StackExchange about what happens when 2 voltage signals are tied together. What’s the resultant voltage and what decides this voltage? The whole train of thought started when I was trying to contemplate what happens when you use pull-ups on signals that are not Open Drain.

I create and simulated a Circuit with the same scenario in LTSpice. “V” is the voltage between the “+” terminals of V1 and V2 and its shown on the right of the simulation. We will confirm the simulation result by doing some math later.

The question is what is the voltage across the load after hooking them up together. And what do the currents look like? Is there a current flowing between the 2 sources as well (apart from the current flowing to the load) because 5v > 1.8v? The simulator refuses to do a simulation without your author adding an internal resistance to the voltage sources first. All voltages sources have certain internal resistances, so that’s fair. This can be considered analogous to having a voltage signal with a certain resistance along its path which limits its current sourcing (or sinking) capabilities.

So I added 1k resistances internally, normally the resistance of a voltage source is far less than this. AA batteries have just 0.1-0.2ohms. Now the circuit looks something like this:

One can simply apply Kirchoff’s current law to the above circuit, the direction of currents would be as in the circuit. I1 and I2 are the currents flowing through R2 and R1 respectively.

By Kirchoff’s law, All the current entering the node labeled V will be equal to the current exiting it even if the currents are not in the actual direction shown above. From this we see:

1
2
3
4
5
6
7
I1 = (1.8 - V) / 1k
I2 = (5 - V)   / 1k
I3 = (V - 0)   / 10k

I3 = I2 + I1
V / 10k  = ((1.8 - V) / 1k) + ((5 - V) / 1k)
V = 3.2381v

Fom this we see the voltage at V is somewhere between 5 and 1.8v. Infact, where it is between 5 and 1.8 depends on how strong or weak the resistances associated with the sources are. If the resistances are lower, then the sources have more of an influence and vice versa. An interesting observation is I1 is negative if you plug V=3.2v in the above equation. This means the current for voltage source V2 (the 1.8v voltage source) is actually flowing into it rather than out of it (its being sinked) and so I1 is actually opposite in direction to the picture shown above.

A simpler case is having 2 voltage sources of the exact same voltage values, in this case the circuit would look like:

Thevenin’s theorem provides an easy simplication into the following, where the equivalent voltage source value is the same but the series resistance is now halved. This results in the following circuit:

Now you can use the Voltage divider concept and easily solve this:

1
2
3
V = V2 * (R2 / (R1 + R2) )
  = 1.8v * ( 10k / (10k + 0.5k) )
  = 1.7142v

As you would notice, the 1k resistance dropped around 0.085v of voltage before getting to the 10k load. Thanks for reading. Please leave your comments or inputs below.

MicroSD Card Remote Switch

| Comments

Recently, I’ve been wanting to remotely be able to program a MicroSD card with a new bootloader or filesystem without removing the card from its embedded target board (such as a Beaglebone or Pandaboard). Due to the lack of any such existing tools, I decided to design my own board. Finally have got it working, below are some pictures and a screencast demo video of the switcher in action! I sprinkled some power and status LED to show the user what’s going on.

The base board requires two SparkFun MicroSD sniffers. The card cage on the sniffer is unused for my purposes. The switcher is controlled through an FTDI cable. I also wrote up a switch program to control the switcher with libftdi. You just have to pass to it the FTDI’s serial number and whether you want to switch to host-mode (for programming the card) or target-mode (for booting the programmed card). Hardware design files are available under a CC-BY-NC-SA 3.0 license.

Screencast

Pictures

Hope you enjoyed it, let me know what yout think in the comments:)