01signal.com

The logic for starting off and resetting an FPGA properly

This is the third and last page in a series about resets in FPGAs. It's recommended to read the first two before this one.

Overview

It's always a good idea to take the time to set up proper logic for creating resets. Even if there's no apparent problem with the design as is, skipping this stage in the design is likely to take its toll: Later on, you might spend days attempting to solve some instability problem, not realizing it's because the logic wasn't initialized properly. In the attempts to resolve problems like this over time, the design accumulates ugly workarounds that are made without understanding the root of the problem. More on this page regarding weird instabilities.

Thinking through and designing the reset signals from the project's beginning is also important in order to assure that it grows properly as new functionalities are added. When an FPGA project is started from scratch, its core functionality is often implemented first, and features are then added as time passes by. As different modules often require separate clocks and resets, it's quite easy to fall in the trap of jotting together something quickly for each part separately. This often makes the project more and more chaotic as it progresses.

It's easier to avoid a messy project by having a central module, that is properly written from day one, for containing resets and clocks. And as further explained below, the reset controller and the clock resources (PLLs and clock buffers as necessary) influence each other, so putting them in the same module pays off. I usually give this module the name clkrst.v.

However it's often not possible to concentrate all of this in one module, as some IP cores, subsystems or design blocks may generate their own group of resets and clocks. When this is the case, it requires careful thinking about what depends on what, and how the overall system should react to requests for resets, that arrive from difference sources. For example, the reset to a PCIe block arrives almost always from the bus itself, and the block produces a reset signal for use by the logic that is connected to this block. In situations like this it's necessary to consider, for example, how the system in general should respond if a reset arrives from the PCIe bus (including the possibility of not responding at all).

Since each project has its own story, there's no single solution for all cases. This page describes concepts and suggests ideas and code snippets which can be used as building blocks for your own reset controller. However, these code snippets are not intended for cut-and-paste directly into a project, but should be treated as demonstrations.

For the sake of simplicity, I'll assume that no other block in the system generates neither clocks nor resets, however extending the controller to the general case is fairly straightforward.

The reset state machine

In most designs, there are two main scenarios that require resetting:

In addition, resetting may be required as watchdog timers expire or a major system failure is detected by some other means.

The expected response to all these scenarios is a fundamental restart, of the sort that ensures that no matter what went wrong, it's rectified. This is often best done with some kind of state machine, that ensures a reset sequence that is consistent and repeatable, no matter the reason it was initiated.

Having said that, there are scenarios where a local, possibly recurring reset makes sense. For example, logic that processes video image frames can be reset before the beginning of each frame. This typically calls for a lightweight mechanism, which boils down to assigning several registers with initial values when a local synchronous reset is activated. This is a reset mechanism like any other, and is often a clean and simple way to ensure robust operation.

But as there's not much more to add about this possibility, the rest of this page focuses on the fundamental reset, which covers the entire FPGA.

Unstable clocks and the need for reset

It's quite common (and usually recommended) to use the FPGA's own PLLs for generating clocks. But these PLLs usually start operating at the same time as all other logic on the FPGA becomes active. As a result, logic that depends on these clocks is fed with an unstable clock, which has a frequency that can be significantly higher than normal.

When this happens, the timing of the relevant logic paths is not guaranteed. Hence any logic that relies on clocks that are generated by PLLs should be treated as if it failed to achieve the timing constraints until the PLL was locked.

The adequate and common solution is to hold all such logic in reset until the related PLL is locked. Alternatively, this logic can be set to ignore the clock until the PLL is locked, e.g. by ensuring that the flip-flops' Clock Enable input (CE) is inactive.

If an external clock is connected directly from the FPGA's pin to its logic elements, the question is whether the clock generator (typically a PLL) has locked quicker than the FPGA was loaded with its bitstream. It's not always easy to be sure about this.

Consequently, many of the synchronous elements need to be reset in most designs. Or more precisely, on the question whether a reset is necessary due to an unstable clock, one can divide these elements into three groups:

It's worth mentioning that some FPGAs allow to set up the configuration process (i.e. the process of loading the bitstream and initializing the FPGA), so that the FPGA's wakeup is delayed until all PLLs are locked. This could be a way to solve this issue. The disadvantage of this solution is that if there is a problem with an external clock, the FPGA will not start at all. A situation like this can be very confusing.

A simple state machine

Since the fundamental start or restart are quite rare events, it doesn't matter if it takes a few microseconds more than necessary. In many cases even up to 100 ms or so is fine, and this can be taken advantage of. A plain counter is therefore a simple way to implement a sequence of events.

In its simplest form, it boils down to something like:

reg [4:0] reset_count;
reg       rst_src_pll, rst_src_shreg, rst_src_debounce;
reg       clear_counter;
reg       master_reset;

initial reset_count = 0;
initial master_reset = 1;
initial clear_counter = 1;

always @(posedge wakeup_clk)
  begin
    clear_counter <= rst_src_pll || rst_src_shreg || rst_src_debounce;

    master_reset <= (reset_count != 31);

    if (clear_counter)
      reset_count <= 0;
    else if (reset_count != 31)
      reset_count <= reset_count + 1;
  end

@rst_src_pll, @rst_src_shreg and @rst_src_debounce represent different reasons to reset the system. These registers are given values by some other logic. I'll go through a few examples of such logic, but for now the important point is that these registers are synchronous with @wakeup_clk (so there is no need for clock domain crossing).

@clear_counter is the logic OR of these reasons for reset. This register changes @reset_count to zero. Otherwise, this counter goes from 0 to 31 (in this example) and stops afterwards.

Finally, @master_reset is active as long as @reset_count hasn't finished counting. This is the simplest form of a reset state machine, and counting to 31 is also quite modest.

So if any of the @rst_src_N signals is active, even for a single clock cycle, the synchronous reset is active for 31 clock cycles.

There are two advantages with a long reset pulse: First, if the related @rst_src_N goes on and off randomly (for example, wobbling PLL lock detectors, push buttons, software that requests resets multiple times), these multiple activations don't propagate to the visible synchronous reset. While such multiple activations are usually harmless, they can result in unnecessary activity of output pins. Such activity can have a negative effect, for example confusing a person that tests the electronics into thinking something is wrong.

In that sense, the example of 31 clock cycles is quite minimalistic. It's even better to count to the value that corresponds to 10-100 ms, if such delay is acceptable. By doing so, any wobbles that are shorter than so are hidden by the reset controller.

The second reason for a long reset pulse is that the distribution of the original synchronous reset into local copies (as explained previously) involves a delay of one clock cycle. If the distribution of the reset throughout the logic requires copying the signal multiple times, the total delay doesn't just get longer, but possibly also uneven. A long reset pulse ensures that at some point in time, all logic is exposed to an active synchronous reset. It's still a bad idea to have uneven delays in the reset path, because the deactivation of the reset becomes uneven as well, but sometimes there's no problem with that. For this matter, 31 clock cycles for the reset pulse is much longer than probably necessary, however it doesn't hurt.

Resets for other clock domains

@master_reset is a regular synchronous reset, but it is synchronized with a clock which may be different from the clocks that are used by the application logic. To produce synchronous resets for other clocks, something like this should be done for each clock:

reg reset_clk_pre1, reset_clk_pre2;
reg reset reset_clk;

always @(posedge clk)
  begin
    reset_clk <= reset_clk_pre2;
    reset_clk_pre2 <= reset_clk_pre1;
    reset_clk_pre1 <= master_reset;
  end

This is just a regular clock domain crossing with three stages, which produces @reset_clk. This signal is the synchronous reset that goes along with @clk. Two stages are actually enough, but since it's an important signal, I've indulged it with an extra register, to be extra safe.

The reset controller's clock

There are in principle three kinds of clocks that can be used as @wakeup_clk, i.e. for the reset state machine:

The first option is easiest to work with. Since there's almost always a reference clock that drives the FPGA's PLLs, this reference clock can often be used directly as the wakeup clock. It's however important to verify that the clock is indeed stable when the FPGA wakes up, i.e. that the time that it takes the FPGA to load the bitstream is longer than the time it takes the external oscillator to produce a valid clock. This is usually the case with a large margin when looking at the datasheets. But if the board's powerup sequence isn't planned properly, it may very well be that the FPGA is given a go-ahead to read its bitstream long before the supply voltage of the oscillator has reached its correct level.

The second option is to use a clock that is generated by a PLL on the FPGA itself. The obvious advantage is that this clock is possibly used for the application logic as well, so it's more efficient with clock resources as well as power. This choice requires holding the reset state machine in its initial state until the clock is stable, which can be done with something like this:

reg rst_src_pll;
reg rst_src_pll_pre;

initial rst_src_pll = 1;
initial rst_src_pll_pre = 1;

always @(posedge wakeup_clk)
  begin
    rst_src_pll <= rst_src_pll_pre;
    rst_src_pll_pre <= !pll_locked;
  end

@pll_locked is the lock detector output of the PLL that generates @wakeup_clk (active high). This signal is asynchronous, so it's first synchronized with @wakeup_clk, and then it's used as one of the reasons for activating @clear_counter, as shown above (i.e. with @rst_src_pll).

Another advantage of this option is that if the reference clock is temporarily unstable or absent (in particular after powering on the board), there's a good chance that the lock detector will be unstable as well. Hence if @reset_count counts to a large number (surely much higher than 31) before deactivating @master_reset, there's a chance that the FPGA will remain firmly in reset until the reference clock is good to work with. This can't be depended upon however.

In any case, the fact that @wakeup_clk is the output of a PLL necessarily means that it feeds logic before this clock has stabilized. It's therefore not clear if the state machine works properly during that time period. It's possible to argue that this doesn't matter, because at some point the clock will become good enough, and proper resets will be generated then. If everything is left in the past after a reset, who cares what happened before that?

A more rigorous approach is that the resets must be held steadily active until the wakeup clock is locked, so the FPGA doesn't behave weirdly soon after its bitstream configuration. This requires paying attention to using only very simple logic with this clock. In other words, the logic should be of the sort that doesn't fail significantly if the clock has a higher frequency than expected.

To analyze what happens if @wakeup_clk temporarily has a too high frequency, note that @reset_count is the only register in the reset state machine that is a vector. This means that all other flip-flops may at worst sample their calculated "next value" one clock too late, because of violated timing. In particular, because @pll_locked is low while the PLL is not locked, @rst_src_pll soon becomes steadily high, and hence @clear_counter is steadily high as well. When the D input (next value) of a flip-flop doesn't change, it doesn't matter how fast the clock is.

The only possible problem is hence with @reset_count, which might count up incorrectly until its calculated next value is held steadily at zero because of @clear_counter. For example, if its current value is 3 (binary 011), its next calculated value is 4 (binary 100). But if the two LSBs aren't sampled because of timing, and the third bit is sampled nevertheless, the counter's value can jump to 7 (111 binary) instead.

To prevent this from happening, the initial values of the chain of registers from @pll_locked to @clear_counter are all assigned in favor of holding @reset_count steadily at zero. Hence if @pll_locked remains steadily low (as it should) until the @wakeup_clk is stable, @reset_count won't move away from zero, and @master_reset remains steadily active.

As for the last option, to use the FPGA's ring oscillator for the reset controller: I haven't tried it myself, so I'm not sure how good idea it is. But if it saves someone who is stuck with no other option, here's more or less how to do it with Xilinx' FPGAs: Look up a primitive called STARTUPE2 (or something like that) in the Configuration User Guide of your FPGA for . It should have an output named CFGMCLK, which is a clock from an inaccurate ring oscillator on the FPGA itself. Its frequency is around 50-65 MHz. This clock is guaranteed to be stable when the FPGA wakes up, and I would set the timing constraint to a considerably higher frequency (say, 100 MHz).

But this is something I would do if there really was no other choice. For example, if the external reference clock isn't stable when the FPGA wakes up, so an extra delay needs to be implemented in logic.

Resetting PLLs

It's generally a good idea to reset the PLLs as part of the reset sequence. This ensures that they get reset when their reference clock is known to be valid. Also, if reset is initiated by the user (e.g. pressing a reset pushbutton), this might be in response to a problem that stems from a poorly locked PLL. Shouldn't happen, but just in case it does.

@clear_counter can't be used to reset the PLL, because it becomes active as soon as a PLL ceases to be locked. If it was used, the PLL would remain in reset state, would never get locked, and the reset would never be released. For the same reason, the PLLs' resets can't be derived from @reset_count: It's held at zero when except for when all PLLs are locked.

The solution is to create a separate reset register for the PLL, which is similar to @clear_counter. So with the notation from above, that PLLs that are not locked are reflected by @rst_src_pll, it boils down to something like this:

reg clear_counter;
reg reset_plls;

initial clear_counter = 1;
initial reset_plls = 1;

assign reset_sources = rst_src_shreg || rst_src_debounce;

always @(posedge wakeup_clk)
  begin
    clear_counter <= rst_src_pll || reset_sources;
    reset_plls <= reset_sources;

[ ... ]

In this code snippet, @rst_src_pll has been excluded from @reset_sources, and is used only for @clear_counter. As a result, the PLLs are reset along with the entire FPGA, except for as the result of themselves not being locked.

Note that if @reset_sources turns low and high randomly, so will @reset_plls according the code that is shown here. This is usually harmless, since nothing bad happens to PLLs when their reset goes on and off like crazy. There's nevertheless a way to avoid this too, as explained next.

More complex startup sequences

The use of a plain counter (@reset_count) as the state variable makes it easy to implement more complicated startup sequences. For example, generating proper asynchronous resets, which are active with the clocks turned off, is quite simple: It's done with simple logic expressions that define the time slots of when the clocks are turned off and when the resets are active.

This plain counter method is therefore a good starting point even for designs that appear to have simple needs from the reset state machine when the project starts. If it turns out later that a complex startup sequence is required, it's easy to expand the existing logic in order to achieve this goal.

In any case, planning the startup sequence boils down to defining the time periods at which each phase in the sequence is in effect. Each such period is translated into a range of values that @reset_count should have.

For example, if the design has two or more unrelated clocks, it's possible to create reset signals for each clock domain as shown above (see "Resets for other clock domains"). But if that method is adopted, each clock domains comes out of reset in effectively random order. This is usually not a problem, but if it is, each clock domain can have its reset deactivated at a defined time, based upon the progress of @reset_count.

It's also possible to implement more than one counter. This is useful when the reset sequence requires waiting for certain conditions to be met before continuing. For example, if the reset sequence involves resetting PLLs, waiting for them to lock, and then continue the reset sequence, it makes sense to have one counter that is held on zero until all PLLs are locked. The second counter is held at zero until the first counter finishes counting.

Note that if a PLL loses its lock, the PLLs themselves will not be reset, but everything that depends on them will. This may not be the desired behavior, as a PLL that loses its lock is a serious fault in most designs. To request a full reset in the event of a loss of lock, add @pll_restart as defined below to the signals that are brought together with an OR to obtain @reset_sources:

assign pll_restart = rst_src_pll && !master_reset;

This simply says: If the PLL became unlocked after the master reset was inactive, reset everything again, including the PLLs. For this to work, the reset count must be long enough to withstand the lock detectors' possible wobbles. In other words, the reset count must be longer than the time the PLL's lock detector can be high even though the PLL isn't locked yet. This is not to be confused with how long it takes for the PLL to acquire lock. Some lock detectors don't wobble at all, and these wobbles are most likely significantly shorter than the PLL's lock time (as specified in the datasheet).

The wakeup shift register

It may be a bit of an overkill, but I usually add a wakeup shift register of this sort in my designs:

reg [15:0]   wakeup_shift;
reg          rst_src_shreg;

initial rst_src_shreg = 1;
initial wakeup_shift = 0;

always @(posedge wakeup_clk)
  begin
    rst_src_shreg <= !wakeup_shift[15];
    wakeup_shift <= { wakeup_shift, 1'b1 };
  end  

Because most FPGAs implement @wakeup_shift as a shift register primitive, which consumes the equivalent of a LUT, it's cheap in resources, and offers another mechanism to make sure a reset takes place during powerup. This is required in a design without PLLs, because nothing else will activate @clear_counter. But even if there are PLLs, there's a possibility that they will already be locked when the FPGA wakes up, as this is a possible option for bitstream configuration on some FPGAs.

So in any case, this is a recommended add-on. Better safe than sorry.

External reset button

Reset buttons are quite common. What they do exactly differs from one design to another. One possibility is that the button that the user considers "reset" is connected to the FPGA pin that kicks off the process of loading the bitstream into the FPGA. It may also be connected to a processor's reset pin, on FPGAs that have an embedded processor.

And this reset button could be connected to a general I/O pin on the FPGA, for the purpose of resetting the FPGA logic. In this case, it's yet another reason to activate @clear_counter.

If @reset_count counts to a number that corresponds to 10 ms or more, there's no need for a debounce of the signal from the input pin, as the wobbles will be absorbed by the counter itself. In this case, this is enough:

reg rst_src_debounce;
reg rst_src_debounce_pre;

initial rst_src_debounce = 1;
initial rst_src_debounce_pre = 1;

always @(posedge wakeup_clk)
  begin
    rst_src_debounce <= rst_src_debounce_pre;
    rst_src_debounce_pre <= reset_button_pin;
  end

However if the counter finishes quickly (as shown in the example above, reaching just 31), the pushbutton needs debouncing. There are several ways to do this, for example:

reg [17:0] debounce_count = 0;
reg        reset_button_d, reset_button_d2, reset_button_d3;

wire       debounce_reached = (debounce_count == 250000);

initial debounce_count = 0;
initial rst_src_debounce = 1;

always @(posedge wakeup_clk)
  begin
    reset_button_d3 <= reset_button_d2;
    reset_button_d2 <= reset_button_d;
    reset_button_d <= reset_button_pin;
	
    if (reset_button_d2 != reset_button_d3)
      debounce_count <= 0;
    else if (!debounce_reached)
      debounce_count <= debounce_count + 1;

    if (debounce_reached)
      rst_src_debounce <= reset_button_d3;
  end

This is a relatively strict debouncer, which doesn't work well with noisy input signals. This is an advantage or disadvantage, depending on your needs.

This code example is written for a 25 MHz clock. The value of @reset_button_pin is copied into @rst_src_debounce if @reset_button_pin has had the same value during 10 ms. Note that @debounce_count changes to zero on the same clock cycle as when @reset_button_d3 changes value. Hence when @reset_button_d3 changes value, it's never copied into @rst_src_debounce immediately, but only after keeping the same value for a long while.

Summary

There are many things to consider when designing the initialization of an FPGA. This page has presented some concepts and ideas, but it's important to keep in mind that the real task is to correctly recognize the events that need to initiate action. It's also important to define the correct response to each such event in order to reliably bring the logic to a working state.

Copyright © 2021-2023. All rights reserved. (ba53f4ca)