You must be coming to the U.S. to direct and develop the E2 visa business.
That does not mean you are working every day in the business. Also, you can travel to your home country often to take care of foreign business interests or an elderly parent, for example.
However, you must be involved in the management, strategy, and finances of the business.