Deep Learning Technology for Object Detection in Libraries

Deep Learning Technology for Object Detection in Libraries
Role I played as
  • Developer
  • Researcher
  • UI / UX Designer
Tools I used
  • Anaconda
  • Google Colab
  • Visual Studio Code
Project duration
  • 1 year 6 months
Platform
  • Windows
  • MacOS
  • Linux
Context

World pandemic

The onset of the global pandemic in early 2020 radically transformed daily life, ushering in a wave of unprecedented restrictions. Governments worldwide imposed strict lockdowns, mandating stay-at-home orders and shuttering businesses deemed non-essential. Social gatherings were banned, and travel came to a virtual standstill as borders closed and flights were grounded. Schools and universities shifted to online learning, disrupting traditional education models. Mask mandates became ubiquitous, altering social interactions and public behavior. These measures aimed to curb the spread of the virus but also brought about significant disruptions to social, economic, and cultural norms.

The problem

In the context of health restrictions, many libraries implemented measures limiting the number of simultaneous visitors. Readers were often required to book a time slot in advance to access the facilities. Once inside, libraries implemented social distancing protocols, limiting the number of people in each section, and encouraging the wearing of face masks. These measures aimed to ensure the safety of visitors while allowing limited access to library resources and services.

Our objective

Based on the background, this project aims to develop a system that combines object recognition and visitor counting, and apply it to the library of Ming Chuan University's Taoyuan campus. From object recognition and real-time counting of visitors, to also detecting prohibited items and violations of internal rules. The counted visitor data is then analyzed to assist librarians in understanding the current situation within the library.

Project timeline
1 year
Research
  • Audit
  • Technology discovery and learning
  • Stack confirmation
  • Project intake and alignment
3 months
Design
  • User interface
  • Feature confirmation
3 months
Development
  • Applying specifications document
Project features
Live footage
Visitor count
Detect and alert suspicious activity
View previous statistics
Export statistics with CSV
Project architecture
Camera recording

Object detection using YOLOv4 algorithm

Collect basic features
Collect feature maps on different layer
Predict object type and coordinate
Filter and validate results

Tracking object using DeepSORT algorithm

Predict futur coordinate using Kalman filter
Mahalanobis distance
Appearance descriptor
Relation between predictions and results
Filter and validate result using Hungarian algorithm

Results logical process and save

Visitor count statistics
Visitor illegal behavior capture
Data export
Project design
Live footage
Live footage
Dashboard most requested feature.

Based on the footage content, people, masks and illegal objects can be identified in real time and outlined in green if allowed and red if it is not.

Once an illegal object is detected, the following alert sound is triggered.

Main features
Main features
More feature on the dashboard.

1. Shows daily statistics with total number of entry today and the number of people in the library.

2. Contains the list of screenshot took when detecting illegal object.
Analyze previous statistics
Analyze previous statistics
Developed for the administrators in order to predict futur visitor activities using previous statistics.

1. Select date (single or range).

2. Control results with 3 buttons (refresh chart, switch chart format and export with CSV format).
Setting
Setting
Customize features for current need.

1. Enable or disable detections of selected object.

2. Change the volume of alert sound when suspicious activity is detected.