While Byoir was on vacation in Europe, Carl Dickey, the firm's general manager and partner, had been called to testify before Congressional investigators concerning Carl Byoir & Associates' alleged hand in Nazi propaganda. In late 1934, Byoir dissolved the tourist account and nullified the contract with three month's notice.
Five years after both the dissolution of the account and partnership with Dickey, Byoir's reputation became a matter of national controversy. One day in 1933, Wright Patman challenged Byoir's credibility and patriotism on the floor of the House of Representatives.
is] the American citizen who rode the first Hitler Trojan Horse of propoganda
into our country."
Patman believed Byoir's contract with agents of the German Propaganda Ministry was an American's effort to disseminate informative material on the New Germany. He stated, "[Byoir is] the American citizen who rode the first Hitler Trojan Horse of propoganda into our country." Patman then called for an investigation by the House Un-American Activities Committee (HUAC) chaired by Representative Martin Dies of Texas.
Byoir denied Patman's allegations of espionage in a New York Timesarticle on May 28, 1940. On June 5th of that year, Patman addressed Byoir's alleged abuse of power as a Lieutenant Colonel in the Army Reserve and cited allegations that Byoir might have gotten access to secret American defense plans with intent to inform Nazi Germany.
Byoir requested an investigation, which was granted and reviewed by the Federal Bureau of Investigation. On July 17, 1940, the U.S. Department of Justice cleared Carl Byoir of any wrongdoing. For the rest of his life, Byoir fought public opinion pegging him as a Jew aligned with Hitler's interests.