The buliding housing ŻIH has a rich history closely linked with Jewish history in Warsaw. The building was designed by Edward Zacharias Eber and built 1928-1936. The neighboring Great Synagogue was built in 1876-1878 according to the blueprint of Leander Marconi. Before the Second World War, the current ŻIH building held the Main Judaic Library, as well as the Institute for Judaic Studies. The library's holdings amounted to 30,000 volumes. Most of these volumes, unfortunately, were lost to the Germans during the war.

From November 16, 1940 until March 1942, the building was contained within the ghetto and became a major center of cultural and social life of ghetto residents. Concerts and various cultural events were held inside. Members of the Oneg Shabbat group (Joy of the Shabbat), under the leadership of Emanuel Ringelblum, met secretly here to discuss their efforts to document life and destruction of Polish Jews in WWII as extensively as possible. The organization titled Jewish Self-Help--the only Jewish organization sanctioned by the Nazi administration--also moved into the building.

In April 1942, two-thousand Jewish deportees from Germany proper were accomodated here. A few months later there were deported to the Treblinka death camp.

That same year the Germans narrowed the ghetto boarders, thus excluding Tłomackie street. After this point, the library and synagogue, including the square in front, were used as warehouses for plundered furniture.

On May 16, 1943, to retaliate against the Jews for the Warsaw ghetto uprising, the Germans blew up the synagogue and set fire to the library building. The traces of the fire are still visible on the floor of the main hall.

The devastated building was handed over to the Central Committee of Polish Jews (CKŻP). After a renovation funded by the Joint American Distribution Committee, the Central Jewish Historical Commission moved in and began working to preserve the memories of Polish Jews and gather the physical remnants of their cultural heritage. In 1947 the Commission assumed the name of the Jewish Historical Institute (ŻIH). The most treasured collection in ŻIH's archival holdings is the Ringelblum Archive retrieved from the ruins of the Warsaw Ghetto in 1946 and 1950.

The building of the Main Judaic Library stands for a symbol of multi-faceted intellectual Jewish life before the war and of the spiritual resistance under the Nazi occupation. It survived the destruction of Warsaw's Jewish community and became a focal point of it's revival in post-1945 Poland.