News
The xDR Challenge 2018 has been successfully finished! Thank you for your participation.
- Oct.3rd,2018
- The winners of the xDR Challenge 2018 have been announced!
- Sep.5th,2018
- The detailed evaluation metrics and submission format have been announced in here!
- Aug.29th,2018
- Dead-lines for final submissions of trajectories and technical description have been extended!
- Added format for the technical description.
- Aug.7th,2018
- Start accepting official application, and the application form is uploaded.
- Aug.1st,2018
- Added BIGLOBE into Sponsers.
- The prizes are enriched thanks to the BIGLOBE.
- Added description of how to carry the devices for PDR.
- July.30th,2018
- Added description of "Current plans for evaluation metrics" and the scheduled date of the final test data disclosure.
- July.5th,2018
- Added description of "Competition's Flow".
- Jun.8th,2018
- Added description of track to "Categories of the competition".
Presentation Slides
Overview (Original Call for Participation)
xDR Challenge for Warehouse Operations 2018 (hereinafter referred to as xDR Challenge 2018) will be held as an sequel competition to
"PDR Challenge in Warehouse Picking" [IPIN2017]. Same as last year’s competition, this year’s competition will be held as an off-site tracking
competition by sharing sensor data measured in actual warehouse. xDR Challenge 2018 will be hosted by PDR benchmarking standardization committee in Japan.
This year, the targets for tracking competition are not only employees walking around the warehouse during the operation, but forklifts driven by employees.
We call dead-reckoning of vehicle as VDR (vehicle dead-reckoning). This year’s competition is renamed as xDR challenge, which means a competition for PDR and VDR.
For the world’s first competition for VDR, we measured sensor data by smartphones attached to the forklifts during the warehouse operation. Additionally, scale of the data measurement also upgraded.
We measured more than 30 employees and 9 forklifts for one week (5 business days). The detailed whole body movement during the warehouse operation is also recorded by motion capture system as well as 3D reconstruction of the warehouse.
They will be shared as the reference data for the competitors to learn what warehouse operations look like.
Data format for sharing sensor data and evaluation metric will be adhered to the last year’s competition [MobiCase2018].
That is, evaluation will be integrated evaluation by considering not only positional error, but all potential evaluation metrics required for the warehouse operations.
Categories of the competition
PDR-Track:
Similar to the previous PDR challenge, sensor-raw data, BLE beacon measurements, partial WMS logs are shared for competitor to estimate workers’ location. The submitted trajectories will be evaluated by an integrated evaluation metric.
VDR-Track:
Similar to the PDR-Track, sensor-raw data, BLE beacon measurements, partial WMS logs are shared for competitor to estimate workers’ location. The submitted trajectories will be evaluated by an integrated evaluation metric.
Japanese announcement PDF is here. (⽇本語の告知PDFはこちら)
Japanese PDF
Reference:
[Presentation on xDR (pp.10-18)]:
Masakatsu Kourogi and Tomohiro Fukuhara, Case studies of IPIN services in Japan: Advanced trials and implementations in service and manufacturing fields
in special session "Value Creation in LBS (Location-Based Services)", IPIN 2017. Slide
[IPIN2017]: The track chairs: Masakatsu Kourogi, Ryosuke Ichikari:
"PDR Challenge in Warehouse Picking",
IPIN 2017 Competition, Track 4: PDR for warehouse picking (off-site) (2017) Web site
[MobiCase2018]: Takeshi Kurata, Ryosuke Ichikari, Ryo Shimomura, Katsuhiko Kaji, Takashi Okuma, Masakatsu Kourogi:
"Making Pier Data Broader and Deeper: PDR Challenge and Virtual Mapping Party",
MobiCASE 2018 (9th EAI International Conference on Mobile Computing, Applications and Services) (2018) Slide
Competition's Flow
How to participate?
Competitors are requested to follow the steps as mentioned below for participating this challenge.
Step1 Request for Admission
During the registration period, contestant of the competitions can register themselves as the competitors of the xDR challenge.
If you have an interest for joining PDR challenge, please send an e-mail to xDR-Challenge2018@aist.go.jp with your self-introduction (at least with your name and institution). Then we will provide the access pass (ID and Password) for downloading sample data. We regard this e-mail as your pre-admission for xDR Challenge2018.
We plan to allow the competitors to hide their real name and affiliation for public.
We will keep their information secret for public.
In this case, the competitors can use a team name instead of using real name and affiliation for joining the competition.
Step2 Downloading Sample Data
If the pre-admission for the xDR challenge is completed, competitors can obtain a password and a user ID for downloading the sample data of the xDR challenge so that they start preparing the competition.
Step3 Application the Competition
If you decide to join the xDR Challenge, you are required to complete real-admission. In order to complete real-admission, you need to submit the application form (The application form can be download from
here). If you decide to join the xDR Challenge, please fill out the application form and send us via e-mail.
After confirmation of your application and final checks of the test data, we will provide you test data for the competition.
Sample data provided for whom they completed pre-admission are not the final data for the competition.
Step4 Access to the Test Data for Final Evaluation
Actual data for the competition will be provided when the test data disclosure period is started. The competitors can obtain another user ID and a password for downloading the actual data for the competition.
Step5 Result Submission
Competitors should submit their result of estimation of the trajectory with pre-determined format which includes x,y coordinate of the target positions and orientation of the targets with timestamps. Definition of exact format for submission will be included in the actual data package described above.
Dead-line for result submission:
7 September 2018 14 September 2018 (Extended!)
Format for result submission: Introduced in 5. Format of the result submission in
this file.
Step6 Submission of Technical Description
Competitors are required to submit the description about their methods and algorithms and state their advantages of the methods and algorithms in the light of the evaluation criteria.
Dead-line: (TBD)19 September 2018 (Fixed!)
Format: Free format, must contain abstract of your algorithm/method (limited to 300 words), figures are welcomed.
Step7 Announcement of Competition Results
The result of the xDR Challenge will be announced during our presentation at the special session
“A Survey on Indoor Localization Competitions” at IPIN 2018.
We may request the winner of the xDR Challenge to give short comment. If the competitors can not attend the IPIN2018, we may ask them to give comments via teleconference or recorded video.
Evaluation metrics
Evaluation metrics are introduced in detaied in this file.
Basically, the concept of evaluation metrics for PDR and VDR will be adhered the PDR Challenge 2017. We newly adopt methods for error evaluation related to error evaluation. In the PDR Challenge 2017, the BLE signals from the tags are always provided. The methods evaluated in Es in the PDR Challenge2017 not pure PDR, but integrated localization. In order to evaluating “pure PDR/VDR” errors, in the xDR Challenge 2018, the BLE signals are partially hided as the BLE unreachable period (BUP in short). The error evaluations are going to be combination of accumulated error evaluation in BUP by E_accum_error and evaluation of magnitude of absolute error of integrated localization system in other periods by E_ave_error.
Similar to the last year, we adopt other metrics for trajectory naturalness and warehouse scenario, as well.
How to carry the device
The employees carried smartphones with pouched for measuring sensor data for PDR. The typical styles for carrying the pouches are shown in the following figures.
Prize
VDR-Track:
PDR-Track:
(a) {
TECCO (equivalent to 100,000 yen made by GOV) +
BL-02 + 150,000 yen} or (b) { 200,000 yen +
BL-02}
Runner-Up:
Important Dates
Competition Testing Period Begins: Mid May, 2018
Competition Testing Period Ends: Mid August, 2018
Final Testing Period (Disclosure of final test data)Begins: 7th August, 2018 ( Admission is required)
Dead-line for Result Data Submission: 7th September, 2018 14th September, 2018 (Extended!)
Dead-line for Techical Description Submission: 19th September, 2018 (Fixed!)
Announcement of Winners: During IPIN2018
Result
Winners
We announced the winners of the competitions at the special session at IPIN2018 as follows:
PDR Track:
Winner : HBSM
Runner-Up : No PDR, No Future!
VDR Track:
Winner :HBSM
Congratulations!
Contact
Ryosuke Ichikari, Ph.D. and Ryo Shimomura
National Institute of Advanced
Industrial Science and Technology
(AIST), Japan
xDR-Challenge2018@aist.go.jp
