Mahatma Gandhi had visited the Khadi and Gramodyog Centre at Badanavalu twice — once in 1927 and again in 1932; Officials say the structures are in a dilapidated condition