IPIN2018 LOGO IPIN2018 logo
xDR (PDR&VDR) Challenge for Warehouse Operations 2018 LOGO

News

We are looking for participants!

Sep.5th,2018
The detailed evaluation metrics and submission format have been announced in here!
Aug.29th,2018
Dead-lines for final submissions of trajectories and technical description have been extended!
Added format for the technical description.
Aug.7th,2018
Start accepting official application, and the application form is uploaded.
Aug.1st,2018
Added BIGLOBE into Sponsers.
The prizes are enriched thanks to the BIGLOBE.
Added description of how to carry the devices for PDR.
July.30th,2018
Added description of "Current plans for evaluation metrics" and the scheduled date of the final test data disclosure.
July.5th,2018
Added description of "Competition's Flow".
Jun.8th,2018
Added description of track to "Categories of the competition".

Overview

xDR Challenge for Warehouse Operations 2018 (hereinafter referred to as xDR Challenge 2018) will be held as an sequel competition to "PDR Challenge in Warehouse Picking" [IPIN2017]. Same as last year’s competition, this year’s competition will be held as an off-site tracking competition by sharing sensor data measured in actual warehouse. xDR Challenge 2018 will be hosted by PDR benchmarking standardization committee in Japan.
This year, the targets for tracking competition are not only employees walking around the warehouse during the operation, but forklifts driven by employees. We call dead-reckoning of vehicle as VDR (vehicle dead-reckoning). This year’s competition is renamed as xDR challenge, which means a competition for PDR and VDR. For the world’s first competition for VDR, we measured sensor data by smartphones attached to the forklifts during the warehouse operation. Additionally, scale of the data measurement also upgraded. We measured more than 30 employees and 9 forklifts for one week (5 business days). The detailed whole body movement during the warehouse operation is also recorded by motion capture system as well as 3D reconstruction of the warehouse. They will be shared as the reference data for the competitors to learn what warehouse operations look like.
Data format for sharing sensor data and evaluation metric will be adhered to the last year’s competition [MobiCase2018]. That is, evaluation will be integrated evaluation by considering not only positional error, but all potential evaluation metrics required for the warehouse operations.

Categories of the competition

PDR-Track:
Similar to the previous PDR challenge, sensor-raw data, BLE beacon measurements, partial WMS logs are shared for competitor to estimate workers’ location. The submitted trajectories will be evaluated by an integrated evaluation metric.  
VDR-Track:
Similar to the PDR-Track, sensor-raw data, BLE beacon measurements, partial WMS logs are shared for competitor to estimate workers’ location. The submitted trajectories will be evaluated by an integrated evaluation metric.
Japanese announcement PDF is here. (⽇本語の告知PDFはこちら)
Japanese PDF

Reference:

[Presentation on xDR (pp.10-18)]:
Masakatsu Kourogi and Tomohiro Fukuhara, Case studies of IPIN services in Japan: Advanced trials and implementations in service and manufacturing fields
in special session "Value Creation in LBS (Location-Based Services)", IPIN 2017. Slide

[IPIN2017]: The track chairs: Masakatsu Kourogi, Ryosuke Ichikari: "PDR Challenge in Warehouse Picking", IPIN 2017 Competition, Track 4: PDR for warehouse picking (off-site) (2017) Web site

[MobiCase2018]: Takeshi Kurata, Ryosuke Ichikari, Ryo Shimomura, Katsuhiko Kaji, Takashi Okuma, Masakatsu Kourogi: "Making Pier Data Broader and Deeper: PDR Challenge and Virtual Mapping Party", MobiCASE 2018 (9th EAI International Conference on Mobile Computing, Applications and Services) (2018) Slide

Competition's Flow

How to participate?
Competitors are requested to follow the steps as mentioned below for participating this challenge.
Step1 Request for Admission
During the registration period, contestant of the competitions can register themselves as the competitors of the xDR challenge. If you have an interest for joining PDR challenge, please send an e-mail to xDR-Challenge2018@aist.go.jp with your self-introduction (at least with your name and institution). Then we will provide the access pass (ID and Password) for downloading sample data. We regard this e-mail as your pre-admission for xDR Challenge2018. We plan to allow the competitors to hide their real name and affiliation for public. We will keep their information secret for public. In this case, the competitors can use a team name instead of using real name and affiliation for joining the competition.
Step2 Downloading Sample Data
If the pre-admission for the xDR challenge is completed, competitors can obtain a password and a user ID for downloading the sample data of the xDR challenge so that they start preparing the competition.
Step3 Application the Competition
If you decide to join the xDR Challenge, you are required to complete real-admission. In order to complete real-admission, you need to submit the application form (The application form can be download from here). If you decide to join the xDR Challenge, please fill out the application form and send us via e-mail. After confirmation of your application and final checks of the test data, we will provide you test data for the competition. Sample data provided for whom they completed pre-admission are not the final data for the competition.
Step4 Access to the Test Data for Final Evaluation
Actual data for the competition will be provided when the test data disclosure period is started. The competitors can obtain another user ID and a password for downloading the actual data for the competition.
Step5 Result Submission
Competitors should submit their result of estimation of the trajectory with pre-determined format which includes x,y coordinate of the target positions and orientation of the targets with timestamps. Definition of exact format for submission will be included in the actual data package described above.
Dead-line for result submission: 7 September 2018 14 September 2018 (Extended!) Format for result submission: Introduced in 5. Format of the result submission in this file.
Step6 Submission of Technical Description
Competitors are required to submit the description about their methods and algorithms and state their advantages of the methods and algorithms in the light of the evaluation criteria.
Dead-line: (TBD)19 September 2018 (Fixed!)
Format: Free format, must contain abstract of your algorithm/method (limited to 300 words), figures are welcomed.
Step7 Announcement of Competition Results
The result of the xDR Challenge will be announced during our presentation at the special session “A Survey on Indoor Localization Competitions” at IPIN 2018. We may request the winner of the xDR Challenge to give short comment. If the competitors can not attend the IPIN2018, we may ask them to give comments via teleconference or recorded video.

Evaluation metrics

Evaluation metrics are introduced in detaied in this file.

Basically, the concept of evaluation metrics for PDR and VDR will be adhered the PDR Challenge 2017. We newly adopt methods for error evaluation related to error evaluation. In the PDR Challenge 2017, the BLE signals from the tags are always provided. The methods evaluated in Es in the PDR Challenge2017 not pure PDR, but integrated localization. In order to evaluating “pure PDR/VDR” errors, in the xDR Challenge 2018, the BLE signals are partially hided as the BLE unreachable period (BUP in short). The error evaluations are going to be combination of accumulated error evaluation in BUP by E_accum_error and evaluation of magnitude of absolute error of integrated localization system in other periods by E_ave_error. Similar to the last year, we adopt other metrics for trajectory naturalness and warehouse scenario, as well.

EvaluationPlan

How to carry the device

The employees carried smartphones with pouched for measuring sensor data for PDR. The typical styles for carrying the pouches are shown in the following figures.

Styles

Prize (tentative)

VDR-Track:
(a) { VDR module (equivalent to 200,000 yen made by Sugihara SEI) + Android IoT device BL-02 (made by BIGLOBE) + 150,000 yen} or (b) { 200,000 yen +BL-02}

VDR module 画像 BL-02 画像


PDR-Track:
(a) {TECCO (equivalent to 100,000 yen made by GOV) + BL-02 + 150,000 yen} or (b) { 200,000 yen + BL-02}

TECCO 画像 BL-02 画像


Runner-Up:
BL-02 + 100,000 yen

BL-02 画像

Important Dates

Competition Testing Period Begins: Mid May, 2018

Competition Testing Period Ends: Mid August, 2018

Final Testing Period (Disclosure of final test data)Begins: 7th August, 2018 (Scheduled, admission is required)

Dead-line for Result Data Submission: 7th September, 2018 14th September, 2018 (Extended!)

Dead-line for Techical Description Submission: 19th September, 2018 (Fixed!)

Announcement of Winners: During IPIN2018

Contact

Ryosuke Ichikari, Ph.D. and Ryo Shimomura
National Institute of Advanced
Industrial Science and Technology
(AIST), Japan
xDR-Challenge2018@aist.go.jp

Sponsors

Supporting communities

Previous challenges

IPIN-2017-PDR-Challenge logo