Calculate exact p-value for F-ratio and Chi Square

How to actually calculate it--if you don't trust computers. 

But you still have to use a computer.

   On a companion page on this website I demonstrated the calculations needed to find the exact p-value for a t-test by hand.  And “by hand” I mean getting the function into a form that I could potentially evaluate with a calculator (but I'm going to ask Maple2020® to actually do the complicated math for me).  I got interested in doing these computations in a statistics class I'm taking under Dr. Mark Payton in my Forensic Science PhD program at Oklahoma State.    

   Here, I work through the calculations to find the exact p-value for an F-ratio derived from an ANOVA.  Again, I wanted to know how to compute the p-value by hand, and not simply how to go look it up in a table. 

   As with my other evaluation, to get started I first turned to the 29^th edition of the CRC Standard Mathematical Tables and Formulae.  Beginning on page 516 we find several pages of "percentage points" of  F-ratios categorized by α-level, and then each individual table is arranged by the two degrees of freedom, m and n. 

   So, where to start?  Well, from our computer ANOVA output, we should have been given the F-ratio of interest.  So, say we find we have F = 3.0.  We also know our degrees of freedom, for this example, let m = 2 and n = 10.

   Go to the first F table on page 516 of the CRC book [below] and find that for degrees of freedom m = 2 and n =10, at the α = 0.10 level, the critical F ratio is 2.92.  Write that down--and note the α level of 0.10.   

Fig. 1. CRC table for critical F numbers at α = 0.10

   Using these tables,

(1) go to the first table, the one calculated for α = 0.10, find our degrees of freedom of 2 and 10 [the organge circles above], see where they intersect [the green circle above], and note the F value found there...for this example, it's 2.92, which is less than our calculated 3.0, so,

(2) keep going until we find an F that's bigger than ours at our degrees of freedom 2 and 10.  Eventually,

(3) we find the two "critical" F-values that "trap" our calculated F-value, and note their associated α-levels.  Remember, again, that's not our calculated F-value from our ANOVA that's in the first and subsequent tables, those are the critical F-values found in the tables for our degrees of freedom and the various α levels. The "critical" F number is the number we will compare to our calculated F number(s) in order to make our decision about statistical significance.

   We know the calculated F-ratio is 3.0, and we know that's bigger than the critical F of 2.92 we just found, so we have to keep going.  Look at the next table on page 517.  Here, for our degrees of freedom m = 2 and n =10, and now α = 0.05, we find a critical F of 4.10, and this one is, in turn, bigger than our F number. So this is where we can stop. Note this one along with it's α level, too. 

  Again, what we want are the two critical F values from the tables that "trap" our calculated 3.0 (at degrees of freedom 2 and 10), and that's the 2.92 and 4.10 numbers from the tables--our calculated 3.0 number falls between those two.

Fig. 2. CRC table for critical F numbers at α = 0.05

   Further, we could look up all the critical F values for our particular degrees of freedom 2 and 10 and construct a table for all the respective α levels--below are all the pertinent α levels provided in my copy of the CRC handbook, other books may have additional tables at other α levels (and keep up with whether your book is giving you upper tail values or lower tail values--you have to keep that straight):

Fig. 3. Table of critical F values for m = 2, n = 10 from the CRC handbook.

   Since the calculated F-ratio of interest, 3.0, falls between the two critical F numbers 2.92 and 4.10, that means the the p-value we want to know is thus somewhere between the respective α levels of 0.10 and 0.05--probably (interpolating “by eye”) closer to the 0.10 end of the scale.  Not really all that helpful in nailing down the exact p-value--to get that I have to do some calculating.

   As before, I captured the function printed in the CRC book using Maple Calculator (an app for iPhone) and uploaded that to my Maple cloud account.  Maple Calculator uses the iPhone camera to capture an image of the written function, uses handwriting recognition, does some pre-processing on the entry, and then uploads it to the cloud.  Very nice. 

   Here’s the function as it appears in Maple2020 after importing it from the CRC handbook and sorting out the details to get it to work:

 Fig. 4.  F function from CRC handbook as entered in Maple2020. 

   Great.  But, as before with calculating the p-values for the t-test, how do I evaluate the Gamma functions if m and n are not even integers?  (If we're calculating Gamma for a positive whole number x, that's just (x - 1) factorial.)

   First look at the denominator—if m and n, the degrees of freedom, are even positive integers, then evaluating those two Gamma functions is very straightforward.  But now look at all the stuff going on up in the numerator—even if m and n are even, x almost certainly will not be--so the result is just not going to wind up being a nice simple integer to work with except in a very few cases, if at all.  I can’t evaluate that Gamma on my own. And if m or n in the demoninator is odd, I can't do those Gammas by hand, either.

   Luckily, one can calculate the exact p-value using integrals (the equivalent integral form of the Gamma function is found on pp. 350-51 of the CRC handbook I'm using):

 Fig. 5. F function with Gamma functions replaced by their equivalent integrals.

   Alright, alright, alright‼!  Now I have a shot at evaluating that by hand--but, again, Maple2020 will kindly handle all the calculations for me instantly.

  The main idea is, though, I have a much better grasp on what's going on in the analysis here than I would have had by just looking at the p-value generated by the statistics software.  And this also demonstrates one of the values of software such as Maple2020.  While I claim I could now (probably) do the math, Maple handles it much better and so much faster and doesn't make silly arithmetic mistakes like I do, and lets me concentrate on "the big picture." 

  Recall our example has F = 3.0 and degrees of freedom m = 2 and n = 10, so I asked Maple to "evaluate at a point" and, upon entering those numbers…

 Fig. 6. Eureka! The p-value.

   Awesome. Notice that, LOL, Maple2020 converted my nice integrals right smack back into those Gamma functions…but that's ok, Maple knows how to deal with 'em! And also note this is the L side value (remember to keep up with whether you're getting upper- or lower-tail values), so will need to subtract this from 1 to get the R side area (or, alternatively, change the limits of the integration).

   If you want to see the actual numbers put into the integrals, here you go...

   Fig. 7.  The numbers from our example as substituted into the integrals and evaluated by Maple2020. 

   We want only the right tail, so remember to subtract that resulting number above in Figs. 6 & 7 from 1 to get the p-value that you actually want: 

1 - 0.9046325684 = 0.0953674316.

   Awesome.  You can easily capture the above images of the equations, if you wish, with Maple Calculator and an iPhone and shoot them right into Maple from the cloud—assuming you have Maple software and a Maple account. 

©2021  Roger D Metcalf.  All worldwide rights reserved.

The computations here were performed using Maple 2020®.

Maple is a trademark of Waterloo Maple, Inc.

References

Maple2020. Maplesoft, a division of Waterloo Maple, Inc., Waterloo, Ontaro.

Percentage points, F-distribution. (1991). CRC Standard mathematical tables and formulae, 29^th edition.  W.H. Beyer, ed. Boca Raton, FL:CRC Press, Inc.

   You may ask, "why the interest in these statistical things?"

   For one thing, I just really like doing statistics.  In my PhD program I actually have a declared minor in Statistics--not many PhD programs have real "minors," but mine does.

   I'm also tacking on computation of the exact p-value for the Chi Square distribution.  It turned out to be almost kind of trivial after getting the calculations of p- for the t-distribution and F-distribution working, so here it is in brief.  

    This is what I wound up with after importing the function into Maple2020 with Maple Calculator as described above:

Fig. 8.  Density function of p-value for Chi Square distribution.

...and, again, I know that's small, but you can click and zoom on the image.  Here, n is degrees of freedom, and x is the Chi Square value.  Maple had a bit of trouble with the form that appeared in the CRC handbook, because the book used the symbol for Chi Square and Maple2020 kept trying to interpret that as x^2.  No worries--I just changed the symbol to plain x in the above and then everything worked fine. --> next column-->

 Maple is a trademark of Waterloo Maple, Inc.

The computations here were performed using Maple 2020®.

References same as above. 

   I was casually browsing my HP-50g user's manual and found that the folks at HP clearly demonstrate how they calculate p-values for t, Normal, F, and Chi Sq distributions respectively:

where m = mean, v = var, n, n1 and n2 are degrees of freedom, and x is your value of interest.  So this is how the HP calculators are doing the math.  Awesome, thank you, HP!

HP 50g / 49g+ / 48gII graphing calculator advanced user’s reference manual. (2009).  Hewlett-Packard Co. pp 3-265--3-267.

   So, here's what Maple said: 

Fig. 9. More math.  Remember, click on the image and zoom.

...and I don't think I want to tackle that with my old HP-67 calculator.  I don't know if I can even do WhittakerM by hand--but I'm pretty sure I can't.

   Instead, again, use the integral form of Gamma, have Maple "evaluate at a point" (n = 5, x [the Chi Sq value] = 6.3),  and there we have it!

   Remember you may need to subtract the result from 1 depending on whether you want upper tail or lower tail, etc.--or, alternatively, just integrate from the Chi Sq value out to ꝏ.

Fig. 10. Beautiful.

   Now that's something that could be calculated by hand.

   Anyway, that's pretty awesome....but look below in the other column to see what I found in an HP-50g calculator owner's manual....

   As I've noted repeatedly, the old HP-21S calculator can do these calculations with, literally, just a couple of keystrokes, and that's why stats folks still love the 21S for quick number crunching with Normal, t, F, and Chi Sq distributions.