The Factory was the hip hangout for artsy types, amphetamine users, and the Warhol superstars. It was famed for its groundbreaking parties. In the studio, Warhol's workers would make silkscreens and lithographs. In 1968, Andy moved the Factory to the sixth floor of the Decker Building, 33 Union Square West, near Max's Kansas City, a club Warhol and his entourage would frequent.

Speaking in 2002, John Cale said "It wasn't called the Factory for nothing. It was where the assembly line for the silkscreens happened. While one person was making a silkscreen, somebody else would be filming a screen test. Every day something new."