Partial Reconfiguration with Vivado How-To


This is the second post in a series of four on Partial Reconfiguration (or Dynamic Function eXchange, DFX) with Xilinx' Vivado. The intention of this post is walk through and explain the steps for enabling Partial Reconfiguration on an FPGA design. It's recommended to read the first post if you haven't done so already, as it explains the concepts behind these steps.

Xilinx rebranded Partial Reconfiguration as "Dynamic Function eXchange" (DFX) in 2020, which is the expression used in Vivado's menus and informative notes. The technical term "Partial Reconfiguration" is used here nevertheless.

For simplicity, this post assumes only one reconfigurable partition in the project. Expanding this to several partitions is fairly straightforward.

The procedure described in this post starts before the project has been enabled for Partial Reconfiguration. So to break it down roughly, it goes:

Preparation for floorplanning

More than any other task, floorplanning is the one that requires brainwork. It's a subtle balance between not wasting FPGA logic area and making sure that both the static and reconfigurable partitions can be placed and routed without any considerable obstacles.

Accordingly, the majority if this post discusses this very topic.

In the distant past, floorplanning was a timing closure technique, helping the tools in placing the logic in a sensible way. As all FPGA vendors' tools have improved, it has been many years since I last saw any timing benefit of forcing a certain placement, as floorplanning does. Today, the best strategy for the purpose of meeting timing is almost always to let the placer decide.

With Partial Reconfiguration there's no choice but to floorplan, so we have to keep in mind not to make things worse. Doing this is a trial-and-error kind of art, but the simplest method to achieve good results is to let Vivado implement the design without floorplanning, and start from how the logic is placed naturally. From that point, try to organize the areas in a way that makes sense with Partial Reconfiguration.

The floorplanning can be updated as the project evolves over time in the plugin usage scenario, but not for Remote Update, i.e. when Partial Reconfiguration is used as a means to make version updates to a design that is released. In the latter case the static part of the design is frozen as soon as the initial bitstream is released, so among others the floorplanning can't change from that moment.

So even before starting with Partial Reconfiguration, the first task is to find a proper FPGA area for the static logic. No point wasting too much time on this, just get the starting point to go on with after the project is split in two pieces.

Don't get confused: The purpose of this first step is not the floorplanning itself, but to see how Vivado places the logic without restrictions, and from that decide which area should be allocated for the static logic. So:

Setting up a project for Partial Reconfiguration

Xilinx' UG909 suggests two flows for working with Partial Reconfiguration:

I'll go with the project flow here, even though it has some limitations, some of which relate to the type of sources in the reconfigurable module (in particular regarding mixing in block designs). Either way, it's better to start off with the project flow, since the scripts that it generates for implementation are a good basis for the non-project flow, if so needed.

These are the steps for turning on Partial Reconfiguration support on an existing project:

Trying to implement the project at this stage will most likely fail with an error saying something like "[DRC HDPR-30] Missing PBLOCK On Reconfigurable Cell: HD.RECONFIGURABLE cell 'pr_block_ins' must have PBLOCK assigned to itself or its descendant cells". In simple words, floorplanning is necessary.


At this stage, the project is set up just enough for the floorplanning task.

Before breaking down this task into small steps, it's worth to mention a few things to keep in mind:

Now breaking it down to steps:

Correcting the floorplanning

This is probably the most daunting part about Partial Reconfiguration: To get the floorplanning just right. If you're into a Remote Update, this is extra important, as this floorplanning will be in force throughout the project's lifetime.

Corrections to the floorplanning are necessary for two main reasons: To address Critical Warnings, and at a later stage to optimize the usage of the FPGA resources: Minimize wastage without creating obstacles for place and route.

On the upside, it's easy to drag a Pblock's boundaries, as well as to expand it with additional rectangles: Right-clicking the Pblock and select "Add Pblock Rectangle".

The Critical Warnings often say what corrections are necessary, and nevertheless be sure to have read the relevant chapter (6, 7 or 8) in Xilinx' user guide, UG909 regarding the floorplanning limitations of your specific FPGA.

The rest of this section discusses possible issues with series-7 FPGA. Ultrascale devices are much easier to floorplan.

One common error with series-7 FPGA is splitting of interconnect tile columns, for example:

[Constraints 18-993] The Pblock pblock_pr_block_ins has defined an area that causes the splitting of interconnect tile columns. Dynamic Function eXchange requires that the left and right paired interconnect tile columns cannot be split by a reconfigurable boundary.  This is caused by either the left or right edge of a Pblock boundary, or by the Pblock spanning over logic types not included in the Pblock ranges.  To avoid an unroutable situation, placement will be prohibited from both of these columns. To avoid placement restrictions, modify the Pblock to avoid splitting the two columns.
The column of the split contains interconnect tile INT_L_X48Y299  (SLICE_X79Y299 SLICE_X78Y299).
Please refer to the Xilinx document on Dynamic Function eXchange.
Resolution: Set the Pblock property SNAPPING_MODE to value of ON, or modify the column/X specification of the pblock to avoid this edge.


[Constraints 18-996] The split between the left and right columns occurs between a reconfigurable Pblock and Static logic. The static sites are not reconfigurable. The Pblock should be adjusted to remove the column from the Pblock, unless the excluded reconfigurable and static sites are not needed for the design. Note that adjusting the Pblock will prevent prohibits and improve placement of the design, but may reduce the routability if the removed sites were needed to span across the static logic. Failure to modify the Pblock may lead to an unplaceable design if these prohibited sites are required by the design. Resolution: Set the Pblock property SNAPPING_MODE to value of ON, or modify the column/X specification of the pblock to avoid this edge. and

To fix this, set the Pblock's SNAPPING_MODE property to ROUTING or ON (odds are that ROUTING won't be good enough, so ON) as suggested in the first warning. Doing this is likely to bombard the XDC file with a lot of constraints of this sort:

set_property PROHIBIT true [get_sites SLICE_X79Y349]
set_property PROHIBIT true [get_sites SLICE_X78Y349]
[ ... ]
set_property PROHIBIT true [get_sites SLICE_X79Y191]
set_property PROHIBIT true [get_sites SLICE_X78Y191]
set_property PROHIBIT true [get_sites PMV_X0Y2]
set_property PROHIBIT true [get_sites SLICE_X36Y190]
set_property PROHIBIT true [get_sites SLICE_X37Y190]
[ ... ]
set_property PROHIBIT true [get_sites SLICE_X79Y176]
set_property PROHIBIT true [get_sites SLICE_X78Y176]
set_property PROHIBIT true [get_sites T14]
set_property PROHIBIT true [get_sites R15]
set_property PROHIBIT true [get_sites XADC_X0Y0]
set_property PROHIBIT true [get_sites SLICE_X36Y175]
set_property PROHIBIT true [get_sites SLICE_X37Y175]
[ ... ]

and it goes on.

The PROHIBIT assignments for slice sites are those silencing the said Critical Warning. The other PROHIBIT assignments indicate logic sites that are included in the geometric region, but are not allowed for Partial Reconfiguration on the targeted device. Ultrascale devices and later produce significantly less PROHIBIT lines, if at all.

For the purpose of just silencing the Critical Warning, it's probably fine to remove all PROHIBIT assignments, and leave one single such for the slices only, covering the range of slices that are listed in Vivado's response to the change in SNAPPING_MODE. So turn it into something like this.

set_property PROHIBIT true [get_sites -range {SLICE_X79Y0 SLICE_X79Y349}]

This is the kind of assignment that may resolve the interconnect splitting issue.

Regardless, a row of this form may also appear in the XDC file, which is apparently fine to remove as well:

set_property HD.PLATFORM_WRAPPER true [get_cells pr_block_ins]

Reducing the XDC file to the minimum that silences Critical Warnings may appear somewhat superficial, but the alternative is a huge constraints file, which is a recipe for confusion later on. In my experience, the absence of such warnings can be taken as an approval that the design is properly floorplanned.

Odds are that returning SNAPPING_MODE property back to OFF will cause problems again, regardless of changes to the XDC.

Adding a reconfigurable module

Up till now, the implementation practically achieves the same as hierarchical design, with some extra restrictions. Even though a partial bitstream is generated, it's quite useless, since loading it retains the same design.

So the goal is to create another partial bitstream, which is based upon another configurable module. This will require adding a Child Implementation.

Be sure to have the previous post fresh in your mind before going on reading this, in particular the part on Parent and Child Implementations, as well as the Dynamic Function eXchange Wizard. Also recall from above on this post, that the "Partition Definitions" tab contains the currently defined reconfigurable modules and their sources.

Open the Dynamic Function eXchange Wizard from the Tools menu, and click Next on the welcome window.

On the Edit Reconfigurable Modules window, click "+". This opens a dialog box for adding a reconfigurable module. The only interesting thing about this dialog box is Reconfigurable Module Name input: This is the name to identify the reconfigurable logic, as already explained above.

The dialog box also requires associating this module with a partition definition name, but there is only one anyhow (under this post's assumption of a single partition).

At least one Verilog / VHDL source file needs to be added to continue: More can be added from the "Partition Definitions" tab later on. Adding the name of the toplevel module in this reconfigurable module won't hurt, in particular if it's not clear from the source files themselves.

Back at the Wizard, click Next again, to the Edit Configurations windows. Click "+", and enter a configuration name. Its only importance is that it appears in the Design Runs window. But just config_2 or so is fine.

A new row appears in the list of configurations. Modify the configurable module in the column for the partition, so that each configuration has a different configurable module.

The last window is Edit Configuration Runs, for assigning runs to configurations. The easy way is to delete all runs listed in this window (if any) and click "automatically create configuration runs". That does what you should do manually anyhow: Create a parent run, call it "impl_1", and then create child runs, call them what you want, and make them children of "impl_1".

The Wizard selects a configuration for each run, however that's easy to change. The only important thing is which configuration gets the parent run.

And by the way, if you delete all runs in the Wizard, all child runs go away, but impl_1 remains.

Finally: Implement the design

To generate the bitstreams, just click "Generate Bitstreams" in Vivado as usual. As already mentioned in the previous post, two or three bitstreams are created when the project is configured for Partial Reconfiguration

For example, on an Ultrascale FPGA the bitfiles can be:

Note that the same number of bitstream files is created for all implementations. In other words, the initial bitstream file is created for the Child Implementation(s) as well, so it's perfectly possible to initially load the FPGA with one of the child implementations, and take it from there.

For a simple way to load partial bitstreams over PCIe or USB 3.x, see this page.

None of the implementation should have Critical Warnings nor fail because of complaints regarding the Pblock and floorplanning in general, because such problem should have already been solved. Should it still happen, the floorplanning must be corrected as explained above.

At times, when clicking "Generate Bitstream", and changes have been made only to the child implementations, Vivado might respond with "Bitstream generation has already completed and is up-to-date. Re-run anyway?". This is somewhat confusing, but clicking "Yes" will run the child implementations properly. This whole thing with Child Implementation is a bit of an add-on to Vivado, which is also why the status during implementation goes something like "write_bitstream complete. Child running".

Reviewing the implementation results

As Partial Reconfiguration is much about placement, it's a good idea to review the implemented designs. You can open a specific implementation by right-clicking "Open Implemented Design" and then hover over the menu item saying "Open Implemented Design" (again). Then select which implementation to open from the list. If an implementation is missing in the list, it's probably already open.

Try right-clicking the top entry of the reconfigurable logic in the Netlist pane of the Implemented Design view, and pick "Highlight Leaf cells". Then do the same with the static logic, with another color.

With the same right-click, there's also "Show Connectivity", which draws straight white lines between logic elements that are connected. The actual routing paths on the FPGA is of course different, so which floorplanning regions these lines cross has no significance. Looking at connectivity can nevertheless help spotting an overall floorplan organization that makes the tools struggle.

It's quite normal that some cells that apparently belong to the static logic are placed inside the reconfigurable area and vice versa, and this is fine. The thing to be wary of is if there seems to be a congestion somewhere — if the logic seems to packed too tight in general, or in some specific region. If possible, changes in the floorplanning can help alleviate that.

Another thing to look at is the positioning of partition pins. They appear as white horizontal bars in the device view, like this (click image to enlarge):

Partition pins in Vivado's device view

As mentioned in the previous post, partition pins can be everywhere inside the reconfigurable partition, however if they are away from its edges, it can indicate that the router struggled with timing during the Parent Implementation.

It's also possible to obtain a textual list of the partition pin's coordinates with this Tcl command (change pr_block_ins to the name of the reconfigurable logic cell):

foreach s [get_pins -of [get_cells pr_block_ins]] { set partpin [get_pplocs -quiet -pins [get_pins $s]] ; puts "$s => $partpin"; }

The coordinates for Partition pins correspond to the grid of CLBs (and not slices). On the GUI drawing, these pins are referred to as "Cell pins".

Some of the cell's pins may not be assigned a partition pin. This happens when there's a mismatch between the reconfigurable module's port list (and/or vector widths) and its instantiation by the static logic module. Such mismatch is perfectly legal (in Verilog), so running this Tcl command can detect unreachable ports, in particular when this is unintentional.

This ends the technical part on setting up the Vivado project, however the next post discusses an important aspect of the FPGA design: How to ensure that the replacement of logic goes through reliably and smoothly.

Copyright © 2021-2022. All rights reserved. (42e6e8c4)