Project Overview
The objective of this project was to design a system to monitor and provide an informational summary of floor conditions over time. The envisioned system was capable of autonomous mobility within a warehouse, capable of capturing images of the concrete floor and automatically detecting any cracks or defects.
Inspections are traditionally carried out by individuals, and the outcomes are largely dependent on the inspector’s personal judgment and experience. This leads to potential inconsistencies in the inspection outcomes across different inspectors and over time. Such inconsistencies proves to be a significant hurdle in the context of asset management and tracking progress, such as the evolution of cracks.
Consistent floor slab inspection is a crucial element of warehouse asset management. To achieve a steady, controllable, and high-quality standard, a transition to automated inspections, potentially employing a robot, is considered advantageous. This allows experienced engineers to devote more of their resources towards the accurate interpretation of results.
Objectives, requirements and stipulations
The automation of floor inspections leads to more uniform quality inspections, and recurrent inspections facilitated thorough monitoring of slab conditions. The primary aim of this innovation is to:
- Enhance inspection quality, yield uniform results, and improve the comprehensiveness of the inspection.
- Better report the condition of the floor slabs.
- Generate an informational summary and deeper insights to ensure swift and rigorous execution of repairs, optimize lifecycle costs, and enhance asset management opportunities.
- Ultimately, devise a system to predict maintenance planning and associated costs for a given floor.
The Complete Narrative: Our Project’s Saga
As we reflect upon our journey, we recall our mission to develop an innovative solution that could potentially revolutionize warehouse inspection and management. It has been a journey filled with learning, collaboration, and progress, which I am eager to share with everyone.
Identifying a Need
Our project originated from a simple observation: manual human inspections of warehouse floors were prone to inconsistencies due to inherent human variability. The aim was to mitigate these inconsistencies and introduce a more standard approach to inspections – a challenge we were ready to confront.
Establishing Objectives
Our vision was a future where the inspection process was seamless, consistent, and less reliant on the individual experience of the inspector. To make this possible, we saw potential in an automated system that could carry out these inspections in an exact, repeatable manner, allowing engineers to concentrate more on analyzing the results.
Establishing Objectives
Our vision was a future where the inspection process was seamless, consistent, and less reliant on the individual experience of the inspector. To make this possible, we saw potential in an automated system that could carry out these inspections in an exact, repeatable manner, allowing engineers to concentrate more on analyzing the results.
Conceptualization and Blueprinting
During the ideation stage, we brainstormed ideas, sought feedback, and planned for a system that could integrate robotics and AI technology to navigate warehouses, photograph floors, and detect any defects.
Getting Our Hands Dirty
The development phase was challenging, but the team met it with passion and dedication. Our objective was to transform our vision into reality, and so we began work, incorporating camera settings, extensive data, point cloud techniques, and artificial intelligence. It was a true test of resilience, teamwork, and problem-solving skills.
Embracing Failure During Innovation
We started with a low-cost global shutter Arducam and a budget autonomous vehicle. We quickly realized that using a robot car could hinder our goals. The robot car’s calibration challenge temporarily shifted our attention away from our main question: Can we effectively scan a significant floor area within a reasonable timeframe while maintaining the necessary precision to detect the targeted cracks? The concrete floors were too slippery for our omnidirectional driven robot: it couldn’t maintain a straight line and veered off track when braking. It was a difficult decision, but we had to let go of our charming robot. We reverted to a handheld trolley.
Next, we attempted to use a laser scanner to detect the cracks. It provided a highly accurate depth view of the surface. However, it was a line scanner, and the area it scanned was skewed when the car didn’t maintain a straight line, which was often the case. So, we had to let go of the laser too. Eventually, we invested in a high-resolution camera. Due to the pandemic and a shortage of electronic boards, we had to wait for almost six months to get a global shutter camera with a resolution high enough to identify the small cracks we targeted.
In the end, the focus remained on our goals.
Figure 1: View of a typical warehouse that needs to be inspected for cracks and other types of defects.
Figure 2: Raspberry Pi with Arducam
Figure 3: The camera set-up is equipped with continuous led lights.
Figure 4: Back to basics: handheld trolly high resolution camera and with two continuous led lights.
Testing, Refinement, and Knowledge Accumulation
Once the system was developed, we transitioned to the testing phase. We subjected the system to rigorous testing within various warehouses. After each run, we improved performance and tweaked the system and AI’s accuracy. It was a period of testing, learning, and adapting, which significantly improved our system. In the last testing phase we were able to run with our car through a warehouse and still obtain images that were accurate enough to detect the cracks on the floor.
Automatic Crack Detection with AI
We used artificial intelligence to automatically identify cracks from images. Initially, a model was trained with a limited number of images. Later, the AI model was trained with open-access datasets that were readily available.
Figure 5: Raw image versus image with detected cracks
Creating a Composite Image by Producing a Point Cloud
The objective of this step was to stitch all the images together, creating a floor overview. We pinpointed the exact location of each picture by automatically deriving locations from the image overlap. We utilized Agisoft to produce the point cloud from the inspection images.
Figure 6: 3D scene reconstruction from photos of the inspected floors.
The Sum of All Parts: The Student Project
After we encountered and overcame issues with the robot and the inexpensive camera, created and tested our handheld car, tweaked the lighting countless times, trained our neural network, managed terabytes of data, created overview images of every floor lane of all the tested warehouses, it was time to bring everything together.
We encountered a group of five students looking for a real-world challenge for their internship assignment. They constructed a fascinating demo application that showcased image results on a floor map. They also introduced a heatmap view of the floor, with red areas indicating more cracks.
Figure 7: floor map of a warehouse with one inspected floor lane, visualized in a heatmap.
Overcoming Obstacles
As with all endeavors, we encountered challenges, including managing budget limitations, resolving technical glitches, and meeting project deadlines. However, each challenge presented a valuable lesson and motivated us to innovate and improve.
The end of a journey
Reflecting upon our journey, we see not just a finished project, but a saga marked with learning, adaptation, and innovation. The experiences and lessons gained from this project have been priceless, and we’re thrilled to apply these lessons to future projects. It serves as a humble reminder of our potential when we collaborate and persistently strive for improvement.
