盖板玻璃划痕检测Cover Glass Scratch Detection

1. 项目背景

该项目面向工业盖板玻璃质检场景，目标是在高分辨率玻璃图像中自动识别并定位划痕、擦伤、灰尘、脏污、夹板、未磨透等瑕疵。此类图像的主要难点在于：原图尺寸很大，单张样例可达到万级像素；缺陷通常呈细长、局部、低对比度形态；淡划痕与背景灰度差异小，容易在去噪或阈值化过程中丢失；玻璃边缘还会出现黑色背景，若直接输入模型，容易引入非玻璃区域干扰。

因此，本项目没有采用单一的整图目标检测路线，而是构建了一个分阶段的工程流程：先将大图切分为可训练、可推理的小块，再用深度学习模型筛选缺陷区域，最后结合图像处理方法提取缺陷轮廓，并把局部坐标恢复到原图坐标系。

2. 数据与任务定义

项目数据覆盖七类区域：擦伤、划痕、夹板、未磨透、灰尘、脏污和正常区域。原始样本分布不均衡，其中正常样本数量远大于缺陷样本，脏污等少数类别样本较少。为了降低类别不平衡带来的训练偏置，训练集制作阶段采用类别均衡采样策略，从每类样本中选取相近数量的图像块参与训练。

3. 总体技术路线

整体方案采用“深度学习分类 + 图像处理分割 + 几何轮廓修正”的混合架构。ResNet 分类模型负责在大量 patch 中筛出可疑缺陷区域，BM3D、阈值分割和霍夫直线检测负责进一步定位缺陷的像素级形态，坐标映射模块负责将 patch 级结果恢复到整张玻璃图上。

1. 对高分辨率原图进行 overlap 切块，保留相邻 patch 的上下文，减少细长划痕被边界截断造成的漏检。
2. 对边缘 patch 进行重采样和镜像 mask 补全，减少黑色背景和非玻璃区域对后续分类的干扰。
3. 使用 ResNet 对 patch 进行缺陷分类，判断局部区域是否存在缺陷以及可能的缺陷类别。
4. 对缺陷 patch 做 BM3D 去噪与灰度阈值分割，提取划痕或瑕疵候选 mask。
5. 使用霍夫直线检测补全断续划痕片段，再进行轮廓提取和坐标回映射，生成整图级可视化结果。

4. 分类模块

分类模块以预处理后的 patch 为输入，采用 ResNet 作为主干网络完成缺陷类别判断。训练阶段加入旋转、翻转、缩放、裁剪等几何增强，以覆盖划痕方向和位置变化；同时使用直方图均衡化、Gamma 校正等对比度增强方法，使细微缺陷在不同亮度条件下仍能形成稳定特征。

分类模块的作用不是直接给出最终轮廓，而是把“整图上的大规模搜索问题”转化为“少量候选 patch 的精细定位问题”。这样既降低了分割阶段的计算量，也减少了在正常区域上进行阈值分割时产生的噪声误检。

5. 分割与轮廓定位模块

对分类出的缺陷 patch，项目进一步执行灰度化、去噪、阈值分割和轮廓提取。早期实验比较了高斯滤波、均值阈值、中值阈值、直方图双峰法和 Otsu 自适应阈值。实验发现，淡划痕区域的灰度响应弱，通用阈值方法容易只分割出局部片段，甚至在噪声较强时出现断裂。

为此，项目采用 BM3D 进行去噪。BM3D 通过块匹配和三维滤波抑制随机噪声，同时相对完整地保留细长边缘结构，更适合低对比度划痕的后续分割。去噪后再根据图像灰度分布设计阈值平面，提取候选 mask。

对于仅被分割出部分区域的浅划痕，项目引入霍夫直线检测作为几何补全手段。通过在参数空间中投票，霍夫变换可以从断续边缘点中恢复潜在直线结构，从而连接不连续的划痕片段。该模块主要用于改善淡划痕断裂、短划痕漏检和轮廓不完整问题。

cv2.HoughLinesP(
    edges,
    rho=1,
    theta=np.pi / 180,
    threshold=18,
    minLineLength=10,
    maxLineGap=120
)

6. 坐标回映射与可视化

集成模块会记录每个 patch 在原图中的左上角坐标、分类结果、分割 mask 和轮廓点。完成局部检测后，将 patch 内的轮廓坐标叠加切块偏移量，恢复为原图坐标，并转换成 labelme 可读的多边形标注格式。这样既能在整图上复核检测结果，也方便后续进行缺陷长度、宽度等几何测量。

7. 项目总结

该项目的关键价值在于把工业质检中的高分辨率、小目标、低对比度问题拆解为多个可控子任务。ResNet 负责稳定筛选候选区域，BM3D 和阈值分割负责提取局部缺陷形态，霍夫直线检测补足淡划痕的几何连续性，PyQt 标注工具和 labelme 结果导出则保证了数据闭环和可验证性。

1. Background

This project targets industrial cover-glass inspection. The goal is to automatically recognize and localize scratches, abrasions, dust, stains, clamping defects, and insufficient polishing in high-resolution glass images. The main challenges are large image size, small local defects, low contrast, fragmented faint scratches, and black edge regions that can interfere with model inference.

Instead of applying a single full-image detector, the project decomposes the problem into patch-level classification and local contour localization, then maps the local predictions back to the original image coordinate system.

2. Data and Task

The dataset contains seven types of regions: abrasion, scratch, clamping defect, insufficient polishing, dust, stain, and normal glass. Because normal samples dominate the dataset, class-balanced sampling is used during dataset construction.

3. Pipeline

The system combines deep-learning classification, image-processing segmentation, and geometric contour refinement. ResNet selects suspicious patches, while BM3D, threshold segmentation, and Hough line detection localize the defect contour.

1. Split the high-resolution source image into overlapped patches.
2. Apply edge resampling and mirrored mask completion for boundary patches.
3. Classify each patch with ResNet to select suspicious regions.
4. Run BM3D denoising and gray-level thresholding on defect patches.
5. Use Hough line detection to connect fragmented scratch segments and export labelme annotations.

4. Classification

The classification module uses ResNet on preprocessed patches. Training applies rotation, flipping, scaling, cropping, histogram equalization, and Gamma correction to improve robustness to scratch direction, location, brightness, and contrast variation.

5. Segmentation and Contour Localization

For suspicious patches, the system performs grayscale conversion, denoising, threshold segmentation, and contour extraction. BM3D is used because it suppresses random noise while preserving thin edge structures better than Gaussian filtering in faint-scratch scenarios.

Hough line detection is introduced to connect fragmented scratch segments. It is especially useful when threshold segmentation only captures partial responses of a faint scratch.

cv2.HoughLinesP(
    edges,
    rho=1,
    theta=np.pi / 180,
    threshold=18,
    minLineLength=10,
    maxLineGap=120
)

6. Remapping and Visualization

The integrated module stores each patch's source-image offset, class result, segmentation mask, and contour points. Local coordinates are shifted back to the original image coordinate system and exported as labelme-readable polygon annotations.

7. Summary

The value of this project lies in decomposing high-resolution, low-contrast industrial inspection into controllable submodules. ResNet provides robust candidate screening; BM3D and thresholding extract local defect structures; Hough line detection improves contour continuity; PyQt annotation and labelme export close the data and verification loop.