Just received the first two pre-production prototypes of the Wireless Debug Probe (WDBP) from MacroFab. After initial inspection and power-up I am so far very pleased with the quality of the work done. There are a few firmware issues to be addressed and I will be publishing the results of the full testing and firmware updates in a future blog post.
For several decades I have designed, built, and tested my own PCBs for several products/projects. During all of that time I made use of traditional “through-hole” mounting components. Beyond the schematic capture and PCB design software no special tools were required for this process other than a good soldering iron and some good quality wire snips to trim the component leads after soldering.
In recent years it has become increasingly difficult to source through-hole mounted components, particularly for the modern microcontrollers that offer the kinds of features new designs require. The time was right to transition to using Surface Mount Technology(SMT) components.
A while ago I began a new project, the design of a Wireless Debug Probe (WDBP) to connect a computer based debugger to and embedded system using either JTAG or SWD. This design, that will be open-sourced upon release, is based up the Blacksphere “Black Magic Probe” (BMP), an excellent open source debugging probe. The BMP itself is very, very small, using some of the smallest available SMT components.
The BMP is about 15mm x 35mm … REALLY small.
Since the WDBP is my first project using SMT components I decided not to use the smallest components so the PCB size is about 35mm x 90mm. While using larger components contributes to the size of WDBP, an additional factor was the Wi-Fi module with integrated antenna. Plus, I chose to add a footprint for the Tag-Connect TC-2050-IDC “bed-of-nails” connector for attaching my debug probe to the WDBP for debugging the firmware:
Here is a prototype of WDBP:
The WDBP prototypes have been built by a contact of mine who has been working with SMT components for some long time so I avoided the need to purchase any specialized assembly tools. However, during the testing there were a few issues that needed addressing, it was at this time that decided to investigate the minimum set of tools that would enable me to rework the prototypes.
The first issue when working with such small components is being able to visually inspect soldered joints. I tried a head-mounted magnifier, however the magnification was just not good enough to efficiently find any soldering issues. This led me to research and purchase of a microscope. The one I chose is from AmScope, model number SW-3T24Z, their Trinocular Stereo Microscope. I chose to get a trinocular ‘scope so that at some point in the future I could add a camera to the setup. Also, after reading the reviews of this microscope being used for SMT I purchased an additional lens to ensure the distance between the lens and the item under inspection was as great as possible to allow tools to be used on the PCB.
Apart from some small tools like anti-static tweezers the remaining tool investment was a hot-air rework station. This station permits a PCB to be held securely while hot air flows over it, allowing components to be removed, or simply reflowed to fix poor solder joints. Again after some extensive Internet searches I chose the Aoyue 866 rework station. This station comes with not only the base rework unit, but also a selection of nozzles for the hot air gun and a temperature controlled soldering iron. It is a good starter kit for anyone setting out on the SMT journey.
So far, the above tools have worked out really well in testing my prototype PCBS. The next phase of the WDBP project is the pre-production prototypes arriving in about 10 days. Having invested in this tool-set I feel confident that any minor issues with the PCBS can be addressed.
In my previous post I wrote about interrupts and atomic operations in embedded systems. After getting bitten myself with just such an issue I thought I would write about the details of what I just experienced on my current project.
First, the background of the issue; the device I am working on connects to a wireless access point to provide an interface to some PC-based software. The issue was that when the PC software started up sometimes the device would appear to drop messages or stop responding to messages. The issue was hard to track down because if the device was run under debugger control the messages appeared to be received just fine. This led me to suspect the code’s logic was being corrupted by interrupts disrupting the management of messages out of the receive buffers.
When a message is received from the network it is placed into a circular buffer, the index of the next free location is updated, and the total number of characters in the buffer is updated. This all happens in the response to an interrupt from the wireless module. Since there was no related interrupt that could be disrupting this I turned my attention to the background task that unpacks the data in the circular buffer into packets for processing. However, what I saw there was my buffer manipulation being protected by the wireless module interrupt being turned off, just as I wrote about previously.
This deepened the mystery since it appeared I was following good practice and keeping the buffer management code atomic, at least to the extent of protecting it to further interrupts from the wireless module, the only other code that manipulated the buffer variables.
The next step was to instrument the two methods that enabled and disable the wireless module interrupt. I did this to make sure that each disable call was matched by an enable call. The results of this test revealed that there were more enable operations than disable operations. After some thinking I realized that the enable and disable methods themselves were not atomic, therefore it was possible for an interrupt to arrive during the execution of the disable call, but before the interrupt had been disabled. What was needed was to track the enable and disable calls and only re-enable them when all callers had requested the re-enable.
To do this, a counter was incremented when interrupt enable was requested and decremented when disable was requested. However, the disable action would only be taken if the counter decremented to zero.
This update appears to have resolved my lost message and non-received message issues.
So, if your system has what appear to be random failures, closely examine the data flow through it and the effects that interrupts may have on that flow. Plus, double check that the code being used to manage the interrupts themselves is clean and does not make assumptions like mine did. Just because a caller requests an interrupt be enabled the code needs to be ensure the action is only taken when each caller that requested a disable has also requested an enable.
I have been trying to track down a particularly tricky problem on my Wireless Debug Probe over the last few days. When embedded systems begin to perform in strange, apparently illogical, ways the culprit is usually related to interrupts disrupting the logic of the non-interrupt, or background tasks.
While this turned out not to be the issue I was facing with the project, the first things checked were the potential impacts of interrupts. The first area of interest was to check if there were any variables or code lines that the compiler may have been optimizing out. In general ‘C/C++’ compilers these days have really good optimizers that remove unnecessary code or reduce it to a minimal number of instructions. The compiler does this by analyzing code in the context of the surrounding code. This can be problematic when for example a variable is being examined in a background task and modified in an interrupt task. A typical use of this may be a boolean flag that gets asserted by an interrupt function and is tested in a background function. As the compiler examines the code around the background testing it may find no other references to the variable and decide it can optimize the test out, or change it in some other way. The result of this is that the background task may never detect the boolean being asserted. Fortunately modern ‘C/C++’ compilers have the “volatile” keyword that allows us to inform the compiler that the code should not be optimized when the variable is accessed.
For a good discussion of the volatile keyword see this article.
A second potential issue is reading and writing of variables being interrupted mid-way through the process by a function that also modifies the variable. When a microcontroller needs to modify a memory-based variable it needs to perform a read, modify, write sequence. This sequence may take several instructions and an interrupt may occur at any point during the sequence. Should the interrupt function modify the variable the value partially read or written by the background function may be corrupted. To avoid such non-atomic operations the background function should temporarily disable any interrupt that may affect the variable to be manipulated, restoring the interrupt state once the operation is completed.
When I started this blog it was partly to track the progress of my latest project, briefly mentioned here. Over the month or two following the original post a number of issues were discovered with the original design. It used a TI wireless module that proved to be problematic in prototype assembly, mostly because all of the external connections were under the module. Added to this the antenna matching components were tiny, smaller than grains of rice.
Needless to say I became concerned that the yield on a production run would be poor and in addition the potential for a lot of rework was higher than I wanted to risk.
A little research revealed a Microchip module that seemed to address both of these issues, the WINC1500 (formally an Atmel part). The package connections are on the edge of the module and the antenna and matching components are on the module itself. The latter also meaning the FCC approval for the module would be transferable to my product.
Since I had not done any breadboard testing of this new module I decided to purchase a couple of breakout boards from Adafruit. The breakout module was put onto an Arduino breadboard and plugged into a Nucleo-F401RE.
This arrangement allowed me to begin testing the hardware interface between the MCU and the wireless module, plus investigate the driver requirements for controlling the module.
The difficulties of getting support for the module were part of a couple of earlier posts. However, the issues were eventually resolved and I moved on the the new PCB design for the project.
As a part of this re-design I decided that I would add the ability to power the device with a battery and to add a battery management and charging circuit. The new PCB was manufactured once again by OSHPark and hand assembled by a technician I have known for many years.
While there were a couple of minor errors in the design and two assembly errors, testing of the new PCB went well and now the firmware is under way once again. I had abandoned writing more code until I had a stable hardware platform.
Just yesterday the device attached to my wireless network and responded to pings from my PC. The next step with the firmware is to create a TCP server and to implement the data link between the PC and the core software.
There is so much new technology to stay abreast of these days that we all need a little help. I find there are a number of YouTube channels that can help with this task.
The Open Source and Maker communities are deep wells of information about new technology and tools for engineers, both professional and makers. One source that keeps a good eye on the crowd souring community is the “MickMake” channel. Here you will find a weekly roundup of new products from several crowd sources:
Another good resource for components and solid tutorials is AdaFruit, in particular their learning section is excellent:
If you are interested in the design process itself and enjoy watching how engineers execute their projects there is a very good channel from Piotr Esden-Tempski. In this channel you will find some captures of some very long live streams done my Piotr. These streams are unedited and provide a great window into the way all projects flow, warts and all. So refreshing after seeing so many how-to videos where the instructor makes everything look way easy and they never make mistakes like us “real folks.” #ibitsy, @esden.
Piotr also has several Gitter rooms where he reports progress on his projects and takes constructive feedback. To get notified when he is live streaming one of his projects sign up for his esden/live room on Gitter – https://gitter.im/esden/live
After having a rather poor experience with the Microchip online support systems (see this post) I finally made contact with a US-based support engineer and in a matter of a couple of days he resolved the issues I was having.
Basically he suggested trying a different driver framework for interface to the WINC1500 module. That driver for the WINC1500 is part of the Microchip Libraries for Applications package. After implementing the new SPI and set up methods the breadboard setup of the evaluation WINC1500 module is looking like it is working.
The next step requires the updated PCB in order to test the wireless functionality and once that is achieved I will be posting the full solution so that others who need to interface between the WINC1500 module and an STM32 device don’t have similar issues … watch this space.
My current project uses an Atmel (Microchip these days) WINC1500 Wi-Fi module. The original part I had chosen for my project proved to be difficult to handle during prototype assembly and this made me nervous for production yield, hence the interest in the WINC1500 range. The schematic and the PCB have been updated with the new hardware, however before committing to PCB fabrication and then assembly of the new design I decided to breadboard the wireless module to be sure I had fully understood the datasheet.
Information on the WINC1500 seemed to be very good, with a source-level host interface package and several long technical documents. To breadboard the device I purchased a break-out board from Adafruit and using an STM32F401RE Nucleo board I was able to quickly get the basic SPI channel working to the WINC1500. However, when I used the published API to initialize the module, the provided code hung up.
I was very pleased to discover that Microchip had an online support-ticket system available and duly completed a new ticket. Imagine my surprise when I got a response, very quickly, from a support technician. The edge came off that quickly when I realized the technician had paid no attention to the extensive description in my original ticket post. Completely ignoring that the SPI channel was set up correctly because the chip identifier was being read correctly, together with many other initialization transactions. The advice received simply listed the SPI settings on the MCU master that I should check.
Sad to report, things have not gotten any better. In spite of spending a couple of days documenting exactly what was happening the technician has failed to address the issue at hand. It is interesting to observe that my ticket was only updated around midnight Mountain Time. This adds to the frustration because it means only one interaction with support each day can occur. The timing of the updates and the name of the technician lead me to believe support is coming from India. I think that if a technology company is going to outsource its support, the very least it could insist upon is coverage during the working day of its major markets. Microchip is a worldwide business and I am surprised they believe that this time-disconnect is acceptable. I am hoping that my case will get escalated, as requested today, and that the poor support provided so far will prove an anomaly.
However, the bigger wish is that Microchip would time-align support staff and the time-zones of their clients.
The USA is one of a few countries that have not fully adopted SI units (sometimes called Metric units) of measurements across the board. Generally for people working in the USA on electronics projects it is a small issue. We quickly learn whatever translation is required. However, sometimes these units do cause problems when the same digits represent different objects.
I was just bitten by this with the footprint of a capacitor on a PCB I was having assembled. I read the data for the capacitor and saw the footprint was “0603”, so I searched my footprint library and dropped in a “0603” footprint. Unfortunately when assembly was attempted the capacitor was way smaller than the footprint. A few minutes of research revealed the issue, what I had used was an “0603” inch footprint, the part was “0603” mm.
Not sure how may other numbers like this exist in both measurement systems for capacitors and resistors, however you can bet that in the future I will be triple checking that!
Isn’t it about time the USA joined the majority of the rest of the world and fully adopted the SI unit system:
Over the next several months I will documenting the development of a new tool for embedded system software debugging. The tool will be launched in 2018 on the Crowd Supply web site. The design, when completed will be Open Source Hardware and Software.
The first couple of stages have already been completed; earlier in 2017 a bench prototype was constructed to enable some basic research to be done. It comprised of a ST Microelectronics Nucleo evaluation board, a Texas Instruments wireless evaluation board and a breadboard to hook it all together.
As is typical of such breadboard prototypes some of the higher speed signals (for example the SPI interface) were a little unreliable; once the basic concept was proven the design of a printed circuit board was begun. The initial design is once again a prototype to provide a stable platform for the completion of the controlling software.
The first PCB is currently being assembled and I will write about the initial testing when it arrives.