Announcement

Collapse
No announcement yet.

Proving PAX validity for 2009 season

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Proving PAX validity for 2009 season

    Before proposing to implement PAX based overall results, we need to evaluate how accurate those factors are and know how much changing track conditions can have an effect.

    Below are the PAX figures for each event this year. I included a trend line that represents the actual times of the filled class winners of each day (by definition in the rules, these are the times that determine the PAX average). The idea is that the closer it matches the slope of the PAX line, the more accurate our PAX numbers are for a given track. Also, the more closely matched car classifications are for the PAX factors.

    The first 8 figures show the normal PAX factors we use for all but MIR (where James' proposed BTPAX was used). There is poorer correlation for DDT (2), Pro (4), and Fabi (7) but they happen to be all similar, with higher classes having the advantage. This suggests BTPAX may be more suitable. Applying the BTPAX to those tracks in figures 9, 10, and 11 show how much better they correlate.

    With the exception of MIR (5), the error between the actual class performances relative to the PAX factors is around 0.2%time/Class or less. This is a percentage of the difference in lap time separating each class. To put that in to perspective, our linear classing spaces each class out about 1.3%time/Class. So in other words, it is saying the PAX factors are separating the classes by 5.8PIPs rather than the ideal 5 PIPs but that is the worst case in 7 out of 8 events. Another perspective is that 0.2%time on a 75s lap is 0.15s.

    PAX does present a problem in Event 5, MIR. The error there was about 0.6%time/Class. Unfortunately, that was the only event I missed so I'm not sure what the conditions were exactly but I am told that it was a very hot day and only 2 sessions were run. Glancing at the timesheets you'll notice that almost 2/3rds of competitors set their fastest times on the last lap of the last session. I t may be assumed that drivers were not able to extract the most out of their cars with the limited track time. The discrepancy at this event is not a big deal in my opinion for 1 out of 8 events but just goes to show you that conditions or schedule throughout a dry event day could affect PAX overall points.

    So what would be the affect of that error on overall PAX based points? Since Event 6 PAX correlated quite well, we can assume that BTPAX is accurate for MIR. With that assumption, in Event 5, the theoretical difference in PAX overall points achieved would be around -2.4 points out of 100 for a competitor in T3 and +2.4 points out of 100 for someone competing in SGT2. We would have to see what the point spread is like after an entire season using PAX to know how significant an impact this would be. For example, if the totals for the top 3 overall are separated by 10 points, then an event or two where we have this large an error wouldn't make a difference in the standings. Since we only count 6 out of 8, it's actually a non-issue unless event 5 was one of only 6 events participated in. I don't have 2009 PAX scores but in 2008, the results were as follows cumulative for best 6 to give an idea of the points spread. 3 points spread the top two and 3rd/4th place were a further 6 points behind.

    2008 PAX Overall Results
    Overall Name Class Total Score
    1 Paul McFadden SGT3 620.820
    2 Chris Paczynski GT2 617.952 ORG
    3 Avi Koifman SGT3 611.428
    4 Jonathan Weir GT1 611.091
    5 Bryan Sperandei GT2 609.148
    6 Mohamed Hussain GT4 606.524
    7 Mark Bothelo SGT3 605.609
    8 Dave Barker GT1 605.504 ORG
    9 Adam Hutchinson SGT1 604.621

    Conclusions:
    PAX and BTPAX values are accurate enough to score the 2009 seasons overall results on. It is quite clear using the above analysis to see which tracks are suited for PAX or BTPAX. Track changes can have an effect on overall points scored where some classes will lose and others gain but it is a small amount. The effect of changing track conditions can be mitigated by changing the order of the run groups so that the same classes are not always running at the same time of day at every event. Track condition changes during the 2009 series would not have affected overall PAX results.

    Also, rain data has not been analysed. It is not a problem if it affects all classes an equal amount. But, if rain favors certain classes over others then we need to have a solution to fairly score each class before we implement PAX. The problem hasn't been defined yet so no solution is being proposed.


    Figure 1


    Figure 2


    Figure 3


    Figure 4


    Figure 5

    Will Chan

    Former OTA CCC Chairman


  • #2
    Re: Proving PAX validity for 2009 season


    Figure 6


    Figure 7


    Figure 8

    Will Chan

    Former OTA CCC Chairman

    Comment


    • #3
      Re: Proving PAX validity for 2009 season

      With BTPAX factors applied to DDT (CCW, no kink), SMP Pro, and Fabi

      Figure 9- PAX Factor error 0.09%/Class


      Figure 10- PAX Factor error 0.20%/Class


      Figure 11- PAX Factor error 0.15%/Class

      Will Chan

      Former OTA CCC Chairman

      Comment


      • #4
        Re: Proving PAX validity for 2009 season

        Good analysis Will.

        I am interested in understanding how PAX was developd.
        What data is used to develop PAX factors?
        Did you use the track records or the top three track records in each class?
        What happens to PAX if the rules are changed and a record setting car would change classes for next year without any changes to the car? Is the PAX adjusted?
        What happens to PAX if a record setting car cheated (car illegal). Was that class record used to develop the PAX factor?
        The BTPAX appears more accurate than PAX. How was the BTPAX developed? What factors were used to adjust for the various tracks?
        Since many of the faster classes are propositioned are you using the raw track record data or are you using the propositioning time to develop the PAX factors? The reason I bring this up is because on many tracks in 2009 the track records were much higher than the lower class record time, so the PAX Line should have been lower for the faster classes. EG. On Shannonville Pro with 12 Events to date, using the class records, these classes are propositioned (GT3, GT1, SGT2, SGT1, Improved, Prepared). Since the higher classes are slower than the lower classes why are they being used to develop PAX factors? Shouldn't you be using the faster times to bring the PAX line down at the faster end of the curve?

        JohnP
        Last edited by John P; 11-11-2009, 01:07 AM.

        Comment


        • #5
          Re: Proving PAX validity for 2009 season

          John,
          This may not be the answer you're expecting, but NO DATA IS USED to determine the PAX factors. This is different than how PAX is calculated in other series, SCCA for example.

          The PAX factors exist as a fundamental basis for seperating the classes. The first page of the rule book talks about Fundamental Assummptions, and one of those is to keep the classes linear with a constant increment. Our PAX factors simply define that increment. When we say that each class should be seperated by about 1s on a 75 sec lap, that's all that PAX is saying.

          Our car classification system and PIP structure is then designed to meet the PAX factors by assigning 5 PIPs to 1 class that should be about 1s/75s lap difference.

          What we are looking at above, is not whether PAX factors match the performance of the cars, but rather, does our car classification system match the PAX factors. I should have titled this thread 'Proving car classification'.

          If the trends don't agree, then it is a problem with our car classification and PIP structure, not the other way around.

          James May interject here if i'm not accurate in my statements. I'm new here

          Will Chan

          Former OTA CCC Chairman

          Comment


          • #6
            Re: Proving PAX validity for 2009 season

            Hi Will,

            I asked the questions about BTPAX because it appears to be a much better fit than PAX, to actuals in 2009 on DDT w/kink, DDT no Kink, PRO, Fabi and MIR. PAX is higher than actuals on faster classes while lower on slower classes resulting in the faster classes being easier to beat PAX while the slower classes have it more difficult to beat PAX.

            Any chance you can do a similar chart comparison by events for 2008 similar to your analysis on 2009 results, to see if the same trend proves out?

            Comment


            • #7
              Re: Proving PAX validity for 2009 season

              Yes, but not by this weekend. We need to look at more years to see where BTPAX is suitable.

              Will Chan

              Former OTA CCC Chairman

              Comment


              • #8
                Re: Proving PAX validity for 2009 season

                So, from the above graphs, going to PAX is a disadvantage to higher classes (mostly above the PAX line), offers an advantage to lower classes (mostly below the line), and the results are non-linear (U-shaped cloud).

                I thought data was to be presented in favour of going to PAX, not the reverse!

                Comment


                • #9
                  Re: Proving PAX validity for 2009 season

                  Nothing favors higher classes that are slower (U-shape results).

                  Mod and underfilled classes don't factor in to the trendlines. Higher classes tend to perform better than average at most tracks with PAX.
                  Last edited by bsclywilly; 11-11-2009, 07:41 PM.

                  Will Chan

                  Former OTA CCC Chairman

                  Comment


                  • #10
                    Re: Proving PAX validity for 2009 season

                    With PAX, it appears the faster classes have an advantage over the slower classes. If the PAX curve is twisted a little down for faster classes and up for slower classes, around the present center point, a fair straight line can be found. Maybe BTPAX is that line.

                    Will, James, what is the difference between these two lines, with respect to how they are developed?

                    Comment


                    • #11
                      Re: Proving PAX validity for 2009 season

                      Cool. Nice work.

                      Comment


                      • #12
                        Re: Proving PAX validity for 2009 season

                        You can search back through the years for the analyses that James did post season years past.

                        The "problem" with PAX charts that you look at here is partially our small sample group. T3 and T2 classes have rarely had strongly competitive drivers and tend to be more of an "entry level" point. SGT 1 and 2 rarely, if ever, had two people in a year consistently driving to the limits of properly prepared cars. Our best sample groups have always been the GT classes up to SGT3 where we've had larger groups of consistent drivers.
                        sigpic

                        Stephen, SPDA VP, OTA Director, CCC Member
                        OTA: SGT1 ! -=- CSCS: SSA #842

                        Comment


                        • #13
                          Re: Proving PAX validity for 2009 season

                          Originally posted by bsclywilly View Post
                          Nothing favors higher classes that are slower (U-shape results).

                          Mod and underfilled classes don't factor in to the trendlines. Higher classes tend to perform better than average at most tracks with PAX.
                          The U shape is historical. The graphs show this year's events; but we've got five(?) years of data. Given you want to look at single events, 2005 Nats (SMP Fabi and Pro) are very representative (dry) events - although I haven't looked at this year's numbers.

                          In any case, the proof is in what PAX (or whatever) would have done to previous years' results, not theory. There must be a perception that the PAX'd results are somehow better/fairer. (If there isn't a change in the results, then there's no reason to change the scoring.)

                          To that end, what to do in the rain? We turn off propositioning when it rains - the data show we have to do this. PAX is a way of more-or-less propositioning everyone, so ... what do you do? I can't see how changing to PAX (or whatever model) until this is answered. I have always been interested in adjusted scoring, but could never think of how rain could be addressed.

                          Also, as an aside, it would be more useful if the graphs were normalized so that the linear equation was y=0 (horizontal line). This would make the differences easier to "see."

                          Comment


                          • #14
                            Re: Proving PAX validity for 2009 season

                            Does anyone know of a rain event in recent history?

                            If PAX scoring is used, we are proposing that scoring for rain events will cap the max score at 100 points. On a normal dry event, those who beat pax average will score greater than 100 points and those who achieve a pax average time will score 100.

                            In the event that there is a rain event, pax average will be calculated normally from the filled-class winners, but you can't get more than 100 points. So if some people got dry runs and some wet, the winners will only be able to achieve the same 100 points.

                            Need to check how this method would work with actual rain results.

                            Will Chan

                            Former OTA CCC Chairman

                            Comment


                            • #15
                              Re: Proving PAX validity for 2009 season

                              First DDT event in 2008 (on the Sunday) IIRC was wet and foggy.
                              -=Brian=-

                              Comment

                              Working...
                              X