Here is a preliminary timeline for completing this task. 

• Evaluate uncertainties in extinction correction for the PS1-Gaia catalog (by early March )
• Reflect the uncertainties of the extinction correction to classification errors (by mid March)
• Validate classification accuracy by using the Gaia simulated catalog (by the end of March)
• Validate classification accuracy by using external catalogs, e.g., APOGEE, MaStar (by mid April) 

Could you elaborate on the 1st-3rd points?  How are you going to estimate the extinction correction uncertainty and then account for it?  Are you going to just increase the photometric error?  Also, I am not sure what the 'Gaia simulated catalog' is (this is simply due to my ignorance).

Thank you for your comments! For points 1-3, I'm thinking to disturb parallax (distance) according to parallax error to estimate E(B-V) uncertainties due to distance errors. We could then adopt E(B-V) uncertainties to extinction-corrected colors for the training set. Using this perturbed training set, we can re-evaluate the classification precision for the training set. We can also adopt such uncertainties to the Gaia simulated catalog to evaluate the classification precision, independently. Gaia simulated catalog is a mock stellar catalog from Gaia archive. This catalog includes photometry, astrometry, and intrinsic properties of stars (Teff, logg, [Fe/H], etc..) simulated based on assumed stellar density distributions + stellar evolution models.      Those are my naive thinking so at the moment I'm not sure if they are totally useful. Any comments or suggestions are appreciated.  

I meant to respond earlier, but apologies for being slow.  The plan sounds OK to me.  Let me make a few more comments/question:
1 – when you estimate the E(B-V) variation from the proper motion distance uncertainty, you implicitly assume that the 3D extinction map is correct (and only the distance to an object is wrong).  There must be systematic errors in the 3D map and the variation E(B-V) is probably a lower limit.
2 – are you going to run a Monte-Carlo simulation to account for the E(B-V) variation in your logistic regression classifier?  I may be wrong here, but I thought the input training sample had only binary classification (i.e., F-star or not) and I was not clear about how you would incorporate the E(B-V) variation.

Thanks for additional comments.

> 1 – when you estimate the E(B-V) variation from the proper motion distance uncertainty, you implicitly assume that the 3D extinction map is correct (and only the distance to an object is wrong).  There must be systematic errors in the 3D map and the variation E(B-V) is probably a lower limit.

That's a good point. Indeed, we should take into account, not only distance uncertainty, but also the uncertainties in 3D dust map. I'm exploring an alternative approach to take into account both. 

>2 – are you going to run a Monte-Carlo simulation to account for the E(B-V) variation in your logistic regression classifier?  I may be wrong here, but I thought the input training sample had only binary classification (i.e., F-star or not) and I was not clear about how you would incorporate the E(B-V) variation.

Sorry that it was not clear in my explanation. I meant doing such Monte-Carlo simulations for the training set and checking how the accuracy and precision change. With this approach, it would not be possible to evaluate the classification uncertainties for individual stars. 

Candidate F-type stars have been selected from PanStarrs1 DR2 and have already sent to Onodera-san to be registered to the target database.