BEANS: A New Projection System

EDIT: If you’re looking for BEANS’ pitcher projections for 2014, they are available here.

As if we needed another foray into the world of projections, I’m doing it anyway. I designed BEANS (Baseball Empirical ANalysis System) largely to project players in the Giants’ minor league system who are oftentimes overlooked by the other projection systems. In doing so, I might have just stumbled upon something that works.

I made various projections for the second half of the 2013 season, which you can find here. Although those projections used the first, rudimentary form of the projection system, they held up extremely well. I went back and compared BEANS’ projections for the rest of 2013 with those of ZiPS and Steamer on the same date, and the results were rather astonishing. BEANS performed on par with Steamer and ahead of ZiPS in projecting 2nd half wRC+ rates for the San Francisco Giants in 2013. Here are some statistics:

 N=18 r r2 RMSE AAE
BEANS 0.53 0.28 33.92 24.24
Steamer 0.54 0.29 34.03 25.95
ZiPS 0.46 0.21 34.93 27.66

Essentially, BEANS performed extremely well relative to Steamer and ZiPS; only Steamer had a better correlation, by a hair, and neither had a better RMSE or Average Absolute Error. However, do note that the sample size is not significant, and it would be foolish to declare BEANS as some sort of magic cure-all. Projections are intended merely to be the “least bad” way to predict future performance, and these results, while promising, have yet to prove that BEANS is less bad.

In any case, encouraged by this result, I moved forward with BEANS, and unleashed a couple more components, in which BEANS would predict WAR by using aging curves on the major components (Batting, Fielding, Baserunning, and the positional adjustment). Now, to see how well the holistic projection system is working, along with some changes to the batting aspect, I did some back testing, where I took a random sample of 20 qualified hitters from 2013 to see how well BEANS would have projected their performance relative to the preseason projections by ZiPS and Steamer. Do note that I am aware that this introduces a bias, in that I am only necessarily testing the players worthy of receiving 3.1 Plate Appearances/Game, and that the sample size is still incredibly small, due to the fact that BEANS is not yet fully automated, and that the 2013 preseason projections for Steamer and ZiPS proved rather hard to locate and convert to a similar form to make for easier comparison.

In the end, the results were similarly sparkling. Note that ZiPS only projected OPS+, not wRC+, and Steamer only projected wOBA, so I compared the former to the end of season OPS+ totals, and converted the latter to wRC+, with the park factor included, to better compare with BEANS’ wRC+ output. Also, Steamer did not post Fielding or WAR projections, so I was only able to compare BEANS with ZiPS.

N=20 r r2 RMSE AAE
BEANS 0.75 0.57 16.64 13.08
ZIPS 0.80 0.65 14.83 12.21
Steamer 0.75 0.57 18.95 10.28
N=20 r r2 RMSE AAE
BEANS 0.67 0.45 7.06 4.75
ZIPS 0.63 0.40 7.50 5.31
N=20 r r2 RMSE AAE
BEANS 0.78 0.61 1.18 0.88
ZIPS 0.46 0.21 1.48 1.28

In wRC+, BEANS performed on par with Steamer and did slightly worse than ZiPS predicted OPS+. It appears as though Steamer made a couple of aggressively incorrect projections in the sample, while BEANS performed more consistently, while both had similar correlations.

BEANS vastly outperformed ZiPS in Fielding and WAR. Dan Szymborski has no reason to fear, however, as the sample size is too small to make any definitive statements. In any case, it appears as if BEANS is barking up the right tree. With that said, it’s time to provide a few projections. As the 2014 season has not arrived yet, here are some preliminary, subject to change, projections for some key players on the Giants. Note that all totals are per 600 plate appearances, so you might expect Hunter Pence to provide 10% more value, and Gregor Blanco 15% less.

Player wRC+ Fielding Positional BSR WAR
Buster Posey 141 3.8 6.7 -4.1 5.8
Hunter Pence 116 0.3 -6.6 2.9 2.9
Brandon Belt 131 3.1 -10.6 -0.2 3.6
Marco Scutaro 90 -3.3 2.4 0.5 1.4
Pablo Sandoval 121 0.5 1.5 -3.1 3.5
Gregor Blanco 87 12.0 -2.9 2.7 2.5
Brandon Crawford 80 5.8 6.9 -1.1 2.0
Angel Pagan 98 -5.4 1.3 6.0 2.2
This entry was posted in Player Analysis, Projections, Roster Analysis and tagged , , , , , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s