Lee vs. Hooker The Utility of Intelligence in the Gettysburg Campaign

With the discovery of a hitherto unseen set of intelligence records in the National Archive in the 1950s, a new dimension was suddenly added to the question of how Robert E. Lee allowed himself to be so ill-prepared before the Battle of Gettysburg, one of the critical engagements of the American Civil War. In addition to the much- debated absence of his cavalry and performance of his generals, additional analysis was required as to whether more credit for the victory was due to a new intelligence structure instituted by the Federals in 1863.

General Lee himself highlighted the importance that he felt intelligence played in the outcome of his Gettysburg Campaign, commenting after the war that the battle had been "commenced in the absence of correct intelligence."¹ His campaign report also left no doubt as to whom he felt was directly responsible for this failure, stating that "The movements of the army preceding the battle of Gettysburg had been much embarrassed by the absence of the cavalry."² These comments fueled the acrimonious dispute between the supporters and accusers of Lee's cavalry chief, James Ewell Brown "Jeb" Stuart, as to whether he had ignored orders and abandoned his commander. However, it would be a hundred years before serious thought was given to the contribution that Federal groups, such as the Bureau of Military Intelligence and US Signal Corps, played in the defeat of the Army of Northern Virginia in that quiet corner of Pennsylvania. While General Lee's comments directed the arguments towards his intelligence agents, any discussion seeking to assess the contribution of intelligence cannot exclude an examination into its utility by the commanding generals.

Most of the Civil War leaders still viewed fighting as a very straightforward affair. Years after the end of the Rebellion, Gen. Ulysses S. Grant would describe warfare as "simple enough. Find out where your enemy is. Get at him as soon as you can. Strike him as hard as you can and as often as you can, and keep moving on."³ This simplistic view of warfare also extended into intelligence operations, since their practice and utility was still firmly rooted in traditional methodology. At the outbreak of hostilities, the acquisition of information as to "where your enemy is" was still overwhelmingly based on human intelligence (HUMINT), but traditional sources such as scout and cavalry reconnaissance was now complemented by new methods like long-range observation from hot-air balloons or signal stations.⁴ The creation of specialist signal corps by both sides, along with the widespread use of the telegraph for information transmission, also gave birth to a new form of intelligence acquisition: signal intelligence (SIGINT). Furthermore, with no comprehensive military intelligence structure in existence before secession, both sides would have the opportunity to craft their own apparatus to supply accurate and timely information for operational and tactical use, provided they could recognise the value of the prize that was in the offing.

By describing the art of finding your enemy as simple, Grant was glossing over the harsh lesson meted out at Shiloh in April 1862, where his army was nearly overrun by a surprise dawn attack from a rebel force camped only two miles away. This would not be the only time that Grant would be the victim of a surprise attack by forces which should have been detected earlier with the application of good intelligence practice. His opinion, however, illustrates one of the key realities of intelligence operations: for all the methodologies of information acquisition and transmission, its utility lies with the ultimate recipient. The best information, even when delivered immediately, is useless if not acted upon correctly. The discovery of Special Order 191 could not persuade the commander of the Army of the Potomac, Gen. George McClellan, to attack Lee's fragmented force once he had pinned a portion of it down on the banks of Antietam Creek in September 1862.⁵ Instead, McClellan dawdled while the separated corps of the Army of Northern Virginia rushed to reunite themselves sufficiently to repel the eventual Union attack.

Good intelligence could not compensate for bad leadership and, as with many facets of battle during the war, how generals viewed and adapted to the changing face of military intelligence would determine its effectiveness in battle. Although the Federal army would eventually be led into battle by Gen. George Meade, it would be the contest between Lee and Meade's predecessor, Joseph "Fighting Joe" Hooker, that would define the intelligence struggle during the Gettysburg Campaign. Assuming command only days before the engagement with the rebels, Meade very wisely chose to retain both Hooker's staff and his revolutionary intelligence apparatus centred on the new Bureau of Military Information (BMI). In the sphere of intelligence, at least, Gettysburg was still very much a battle between the systems of Lee and Hooker.

Maj. Gen. Joseph Hooker. National Archives and Records Administration.

Although much of his acclaim would be rightly based on his willingness to tear up the rule book of prevailing military teaching, Lee was thoroughly orthodox and indeed conservative in his utilisation of military intelligence. The US Army, which had developed to tackle external enemies rather than internal, evolved a structure whereby a commanding officer would be expected to manage the acquisition of tactical information from his own resources. The cornerstones of intelligence planning were, therefore, centred on the US Corps of Engineers and the cavalry. The Corps of Engineers was a relatively new addition to the army, but the roots of its practice could be traced back to the art of geometric warfare schooled in West Point. During the Mexican War of the 1840s, the engineers were prominent in coordinating logistics and designing the plans for tackling the formidable local fortifications. Lee had been one of General-in-Chief Winfield Scott's senior engineering staff and so an agent of intelligence—and a particularly good one. While Mexico City would fall within sixteen months of the outbreak of war, the legacy of West Point and the experience of the Corps of Engineers would permeate throughout the later Civil War. Lee and his engineering peers represented an extraordinary number of senior generals on both sides and their traditionalist views on military intelligence acquisition and utility would dominate operations throughout the war.⁶

Vital as the engineers became in Mexico, it would be the cavalry who developed into the most important weapon in both intelligence and counterintelligence operations. In the words of one of Lee's staff officers, Walter Taylor, "An army without cavalry in a strange and hostile country is as a man deprived of his eyesight and beset by enemies; he may be ever so brave and strong, but he cannot intelligently administer a single effective blow."⁷ Never adopted as a significant instrument of battle as it had been in Europe, the principal function of the cavalry in North America was one of raiding and reconnaissance. The undisputed champion in the Eastern Theatre was Confederate Gen. Jeb Stuart, the Army of Northern Virginia's young, ebullient chief cavalry officer. His abilities in both observing the enemy army and stopping them from observing his army even caused opponents such as Gen. John Sedgwick to credit him as "the best cavalryman ever foaled in North America."⁸ Although Lee knew Stuart before the war, it was Stuart's spectacular ride around McClellan's army immediately before the Seven Days Battle of 1862 that forged the special bond between him and his commander.⁹ Whenever Lee needed information it would be Jeb Stuart to whom he would turn.

Col. Walter Taylor, one of Gen. Robert E. Lee's staff officers assisted with intelligence activities. US Army Heritage and Education Center.

Moulded to suit operations in hostile terrain like Mexico or the Indian territories, the intelligence structure of the US Army utilised by Lee and many of his traditionalist peers was overwhelmingly biased towards the acquisition of basic HUMINT, underpinned by cavalry reconnaissance. Beyond this staple, the rebel general sought little additional tactical information except for the latest reports offered up by the Northern press and his War Department in Richmond, spiced by titbits from a small network of professional spies in Federal territory. This information would usually be received raw and unverified with only minimum assistance available from his staff officers, none of whom were dedicated to the processing of military intelligence. During the Gettysburg Campaign, his staff of less than fifteen men processed over 4,000 documents as part of their duties in coordinating and running the army so Lee would be forced to act as his own analyst. Prior to launching his most daring and hazardous operation of the war, which would see him invading Pennsylvania seeking a fight on Union soil, Lee's only meaningful effort to improve his intelligence capacity before his march was to bolster his cavalry.

Alan Pinkerton, a Chicago detective, McClellan's intelligence chief who consistently overestimated the strength of the Army of Northern Virginia. Library of Congress.

While it might seem myopic of such a great tactician to base his intelligence operations around limited HUMINT acquisition and a small staff, this was exactly the structure he desired. His view of the utility of intelligence very much reflected his laissez-faire style of command, which saw him unwilling to retain control over his generals during engagements. Instead, he was content to issue orders at the start of the day and let his commanders execute them as they saw fit. As he described it, "I leave the matter to God and the subordinate officers."¹⁰ As Lee saw it, in this new reality of armies numbering tens of thousands, any attempt to coordinate personally the attacks on the field would be logistically impossible and would, at the least, result in unhealthily slow troop movements. Instead, a heavy responsibility was placed on his corps commanders to enact his orders and therefore Lee saw the priority to be a restructuring of his army from two corps into three, rather than to improve his intelligence capability. Since he believed that an army needed to be self-sufficient, in all likelihood, even if the Confederacy had instituted a centralised service dedicated to intelligence operations, Lee probably would not have used it.

Brig. Gen. Daniel Butterfield, Hooker's Chief of Staff. National Archives and Records Administration.

While Lee's legacy is that of the tactical master, Joseph Hooker is seen as little more than an abject failure; yet another Union general to assume command of the Army of the Potomac amid much bluster only to melt in the crucible of Lee's genius. Obscured by his battlefield blundering and fast lifestyle, however, was the work of a genuine visionary. Having seen more than his fair share of blood on the fields of Antietam and Fredericksburg, Hooker, more than any other general in the war, understood the need for good military intelligence and the shocking neglect shown by the Federal command in developing a system that could support him. Like the Confederacy, Lincoln's War Department lacked any interest in the generation of tactical intelligence and instead the generals in the field were left in charge of their own requirements. With the disbanding of Alan Pinkerton's "Federal Secret Service" following the removal of McClellan from command, and the total lack of regard for intelligence shown by his successor General Ambrose Burnside, Daniel Butterfield, Hooker's chief of staff, would later declare:

[W]hen General Hooker assumed command of the army, there was not a record or document of any kind at headquarters of the army that gave any information at all in regard to the enemy. There was no means, no organization, and no apparent effort to obtain such information. And we were almost as ignorant of the enemy in our immediate front as if they had been in China.¹¹

This realisation drove Hooker to embrace all forms of intelligence acquisition and break away from narrow traditionalist thinking. In the field of intelligence, it was Hooker who would deserve the

Col. George H. Sharpe, head of intelligence for the Army of the Potomac. National Archives and Records Administration.

acclaim as the Civil War's brightest star, while Lee would be revealed as narrow-minded and traditional. The lynchpin of Hooker's vision would be a coherent and centralised intelligence apparatus that was finally fit for the purpose of gathering and interpreting information and processing it into useable intelligence for the commander. At its heart was the BMI, established in February 1863 under Provost Marshal Marsena Patrick and led by a former lawyer, Col. George Sharpe. The BMI received all information brought in by or made available to the army for verification and distillation into a concise form. This was the foundation of the first "all source" intelligence organ in American military history and not only would it run its own scouts and spies, but it would accept and process any HUMINT or SIGINT it could get its hands on. This system allowed the BMI to remain independent and maintain the same utility irrespective of the changing operational or tactical conditions. Furthermore, the more information it was fed, the more accurate it became.

When looking for a demonstration as to the effectiveness of the two contrasting intelligence systems, one need only examine the events of June 28, 1863, three days before Gen. Henry Heth marched his division of Lee's army down the Chambersburg Pike toward Gettysburg. The Army of Northern Virginia was now fully across the Potomac River with no sign of pursuit from Fighting Joe. At his camp near Chambersburg, Lee articulated his satisfaction with their progress to Isaac Trimble:

When they [the Federals] hear where we are they will make forced marches to interpose their forces between us and Baltimore and Philadelphia. They will come up … broken down with hunger and hard marching … strung out on a long line and much demoralized. I shall throw an overwhelming force on their advance, crush it … and virtually destroy the army.¹²

The only note of concern was the mystery surrounding the whereabouts of Jeb Stuart. Four days earlier, and starved of quality intelligence on Union movements due to the incessant attacks of the Federal cavalry, Stuart had been let loose into Maryland to gather information, disrupt Hooker's pursuit and link up with Gen. Dick Ewell's Second Corps further north. This was just the type of marauding, high-risk intelligence mission that Stuart reveled in. Lee was so confident in Jeb's ability that he ordered the rest of his available cavalry, under Gens. Beverly Robertson and William "Grumble" Jones, to stay in Virginia and cover the gaps in the Blue Ridge Mountains. Now, with still no word from Stuart, Lee was proceeding with his plan for the concentration of his divided army on Harrisburg on the banks of the Susquehanna.

Despite having marched past the scene of the most horrific single day's fighting in American history only days before, the bloody climax of the last Confederate march north, Lee had sought no other remedy to his intelligence problem than the reports of his chief cavalry officer. In truth, Lee had very little other option, since he was now operating so far from Richmond that he was all but cut off from his War Department, while Fighting Joe had clamped down on the media. His total confidence, and dependence, on Stuart must have made it all the more shocking when a scruffy, but well-dressed Southern civilian arrived unannounced from Gen. James Longstreet's camp on that evening. The visitor, the spy Henry Harrison, was unquestionably one of the most extraordinary figures in the entire Gettysburg Campaign. Sent to Washington about a month earlier to bring back "information of importance," Harrison managed to navigate the Union lines and locate the camp of Longstreet's First Corps with extraordinary ease.¹³ None of the other private spies dispatched by the rebels was so successful, but having delivered his news, Harrison (almost) disappeared from the story completely. His report was shocking—not only had the Army of the Potomac moved from its position shielding Washington, but it had already crossed the Potomac River and was now only fifty miles away. But there was more: General Hooker had resigned as head of the army and had been replaced by George Meade.

One can only speculate as to Lee's reaction. The enemy were only two day's march from him and so his plans to continue on and bring his army together at Harrisburg would have to be scrapped. He now needed to prepare for battle. Within hours, orders were sent out to Ewell and A. P. Hill to come together to the east of South Mountain, between Chambersburg and Gettysburg. As Union Gen. Abner Doubleday later wrote, "The head of the serpent faced about as soon as its tail was trodden upon."¹⁴ Just as concerning was that if the Federal army had been on the march and he had not heard about it, Stuart must be either dead, captured, or marooned in Maryland or Pennsylvania. Whatever the reality, Lee had to assume that he would be without the majority of his cavalry for the next few days while Robertson and Jones marched up from Virginia and so he would be blind to any rapid advance by Meade.¹⁵

Men of the Bureau of Military Information including, from left: Col. George H. Sharpe, John C. Babcock, unknown, Lt. Col. John McEntee. Library of Congress.

If Lee was to complain of a lack of good information, Hooker would have the opposite problem. The flow of data that had been feeding the BMI since the first of Lee's men left Fredericksburg in early June would slow to a trickle as the Army of Northern Virginia moved into the Shenandoah Valley, only to become a flood as it reappeared to cross the Potomac. As he moved into Pennsylvania, Lee marched into a web of Union intelligence with the BMI and Hooker at its centre. Many commentators have criticised Fighting Joe for dithering in the middle part of June 1863, as if bewitched by Lee's elaborate operation. Certainly the Confederates exploited every opportunity to obscure his operation using the natural cover of the Blue Ridge Mountains to funnel the army safely up the "Great Highway of Invasion," while Stuart foiled almost all Union attempts to monitor its progress.¹⁶ Furthermore, as Ewell's corps and John Imboden's cavalry marched ahead to menace Harrisburg, the commander of the newly formed Department of the Susquehanna, Darius Couch, began to deluge Washington with hysterical distress calls.¹⁷ So exaggerated were some of the reports of the rebel advance that by mid-June Daniel Butterfield would complain that "Cavalry enough is reported to have appeared to fill up the whole of Pennsylvania and leave no room for the inhabitants."¹⁸

Gen. Marsena R. Patrick, Hooker's Provost Marshal. National Archives and Records Administration.

Calm amidst the storm of contradictory and unverified information that flew between the Army and the War Department, George Sharpe and his team tirelessly processed the data, while also coordinating their own intelligence operations. Hooker was by no means immune to the stress of the situation and he would vent his frustration on his intelligence agents, leading Provost Marshal Marsena Patrick to remark that "He [Hooker] has treated our 'Secret Service Department' (headed by Col. Sharpe) which has furnished him with the most astonishingly correct information with indifference at first, & now with insult."¹⁹ Eventually, however, convinced by the weight of information indicating that Lee was headed for Pennsylvania, the order was dispatched on June 25 to move the army north of the Potomac. Coming as it did just after Jeb Stuart had crossed their lines, this decisive movement had the unexpected effect of cutting him off from the rest of the Army of Northern Virginia, thus ensuring that Lee would not find out about it.

Whether one adopts the view that Hooker froze while Lee marched past him or not, a clear distinction should be made between the utility of the intelligence service that Hooker had created and his own usage of it. By separating the duty of information analysis from the military staff and into its own department, he had created the opposite environment to that of Lee. Hooker was spared the additional burden of intelligence work and, conversely, its staff were largely free from his interference. Lee, on the other hand, placed himself at the centre of his own intelligence gathering, putting an artificial constraint on these operations. It would not have been possible for him to have coped with the volume of information which the Army of the Potomac received during June 1863, but more damaging was Lee's own preference for information acquisition. The reason that Lee did not call up Robertson and Grumble Jones to fill in for Stuart during his ride was simply that he did not trust them as he did Stuart. His army was left blind because he ordered away his only genuine intelligence asset. Hooker, on the other hand, could expect to enjoy every form of information available and ultimately no one would benefit more from this system than George Meade.

As the commander of the Federal Fifth Corps awoke early on June 28 to see Col. James Hardie standing over him, fresh from Washington, Meade's first thought was that he must have been relieved of duty. Little did he or almost anyone else in the army know that the previous days had seen the climax of a long-running battle between Washington and Joe Hooker over control of the Army of the Potomac. A spat over Washington's refusal to allow him to access the soldiers of the Middle Department around Harper's Ferry quickly brought Hooker to the boil and he tendered his resignation on June 27. In truth, Lincoln and his War Department had been waiting for this chance since Fighting Joe had so badly bungled his Chancellorsville Campaign two months earlier and it would take them only a few hours to grasp the opportunity. All this was an irrelevance to Meade, who returned from his subsequent meeting with Hooker to greet his son with the words, "Well George, I am in command of the Army of the Potomac."²⁰

Once the shock of his appointment had subsided, Meade was faced with one immediate and pressing problem: he had almost no idea where the various corps of his own army were located, let alone those of Lee. Hooker's obsession for generating positive intelligence on the enemy was matched only by his paranoid demand for secrecy as to his own plans. To ensure no similar leaks as those which infected his Chancellorsville Campaign, Hooker decided not even to share information with his own generals.²¹ "General Hooker, left the camp in a very few hours after I relieved him," Meade later testified to the Congressional Joint Committee for the Conduct of the War, "I received from him no intimation of any plan, or any views that he may have had up to that moment."²² In normal circumstance, such a dramatic change of leadership while the army was spread wide searching for the enemy would be potentially disastrous, let alone when the new commander had almost no understanding of the tactical situation. Fortunately, the true beauty of the utility of Fighting Joe's intelligence system was about to reveal itself, but it was a sad irony that it would take his own removal to prove it. Having wisely decided to swallow his distaste for Daniel Butterfield and retain Hooker's staff, Meade set about absorbing the intelligence reports of the BMI Within twenty-four hours of assuming command in a position of almost total ignorance, Meade issued his first orders for action.

Two days after Meade's appointment, the soldiers of Gen. James Pettigrew's division of the Confederate Third Corps exchanged fire with Federals on the outskirts of the small town of Gettysburg. Already on edge following the news that the Army of the Potomac was in pursuit, Pettigrew received an ominous warning from none other than Henry Harrison as he approached the town. According to the spy, Gettysburg was full of cavalry from the Union First Corps. When he reported the incident to Gens. Henry Heth and A. P. Hill, however, Pettigrew's concerns were dismissed: these were just local militia and Lee himself had confirmed the closest Yankees were still twenty miles away at Middleburg. Hill ordered Heth and the rest of the Third Corps to march into Gettysburg the following day, July 1. Their comrades of the Second Corps marched to join them from the north while Lee and Longstreet followed from the west. Hill was asked if there were there any reasons why they could not advance on Gettysburg tomorrow. "None in the world," was his reply.²³

In contrast to the "spirit of disbelief" that infused the Confederates, John Buford's cavalry division was in no doubt as to the crisis.²⁴ Thrown forward by Gen. John Reynolds, who had been given charge of three of the Union corps in order to expedite the hunt for Lee, Buford was accompanied by BMI scout Edward Hopkins and members of the US Signal Corps. Coordinating with a network of local scouts, who reported to Darius Couch but whose information still fed into Hooker's intelligence machine via Washington, Buford and Reynolds were in no doubt that the rebels were approaching Gettysburg. The small action of June 30 confirmed Buford's suspicions and during that evening he moved his men about four miles forward of the town, understanding that if Hill's men returned he would need to hold them long enough for Reynolds to support him. Likewise, Reynolds knew that if Bu-ford made contact, it would be a race against time before Lee forced him out of the town and potentially seized the high ground to the south, making any subsequent Federal attack considerably more difficult. While Lee's corps were already consolidating, the Army of the Potomac was still fanned out across a radius of over thirty miles and Meade, ever the engineer, was eyeing the promising defensive position of Pipe Creek. It would take at least twenty-four hours to bring them together and so there could be no hesitation if they were to make their stand. Within hours of first contact on July 1, Reynolds was at Buford's side and preparing an urgent message for Meade. In one of his final acts before being shot down by a rebel sharpshooter, Reynolds instructed his courier, "Don't spare your horse, never mind if you kill him."²⁵ The Battle of Gettysburg could not realistically have been won on July 1 by the Federals, but it could have been lost. On receiving his message, Meade abandoned his "Pipe dreams"²⁶ and rushed his army to the scene. Crucial to their establishment of a strong position, before the Confederates could seize the advantage, was the fierceness of their defence in the opening hours and the speed at which the army came together once committed to the battle.

Brig. Gen. John Buford. National Archives and Records Administration.

Lee correctly identified intelligence as one of the critical factors in his eventual defeat, but with his simplistic view of its utility it is hardly surprising that the great Confederate general could not see his own culpability. The Confederacy's most daring campaign finally reached its climax with its most illustrious general sending his men blindly on to an enemy that was already preparing itself to receive him. He had created a system with no versatility, which was rendered effectively ineffective as soon as the Federals began marching toward the Potomac and cut off Stuart. Conversely, John Reynolds, and in turn George Meade, understood exactly how critical the first hours of the engagement would be, thanks to the processed intelligence furnished by Hooker's comprehensive system. While few historians would dare put Hooker's name into the same class as Robert E. Lee, in the field of intelligence it was Fighting Joe who deserved the acclaim as the Civil War's brightest star. Lee would lose the battle to George Meade, but he had already lost the campaign to Joe Hooker.