BJP's Vishwajit Rane retained the Valpoi seat from which he had been elected earlier this year on a Congress ticket only to resign and join the BJP.
Some party leaders, however, said people in the area wanted to teach a lesson to BJP's Ved Prakash for quitting the position in March this year.
"Congrats to all the dedicated volunteers of AAP for Bawana ByPoll win and all the best to CYSS Raj Gopal Rai you deserves a bow", Kumar tweeted. AAP won the by-polls with more than 24,000 votes and the party got total 59,886 votes.
Counting of votes in four assembly constituencies - Bawana in Delhi, Nandyal in Andhra Pradesh, Valpoi and Panaji in Goa got underway on Monday morning.
Congress' Surender Kumar, a three-time MLA, finished third though the party sought to find solace in doubling its votes from around 14,000 to 31,919 votes.
Bawana is one of the biggest assembly constituencies of Delhi.
Sources in the Delhi Congress said the party had expected to gain from the AAP MLA, Ved Prakash, defecting to the BJP and contesting as its candidate.
Almost 75 per cent voting was recorded during the August 23 bypolls in Goa. Soon after the February 2017 polls, Rane resigned his seat just ahead of the floor test, blaming the Congress leadership in Delhi as well as Panaji for failing to make an attempt to form the government despite winning 17 seats.
"As the president of Delhi BJP, I accept the mandate and take responsibility for it", Mr Tiwari told reporters in Delhi, adding that the result was against the expectations of the party.
The electoral fate of Goa Chief Minister Manohar Parrikar, who contested the assembly by-election from Panaji, is also to be decided. They had urged the Election Commission to use VVPAT machines for tallying votes. Prime Minister Narendra Modi congratulated Mr Parrikar on his win.
The by-polls in Nandyal constituency in Andhra Pradesh was a battle of dignity between the ruling Telugu Desam Party (TDP) and main opposition YSR Congress Party, both taking the fight to the next level.
After the setback in Delhi, relief to BJP came from Goa, where the party managed to win both Panaji and Valpoi seats.
The fourth by-poll - for the Nandyal seat in Andhra Pradesh - nearly as keenly contested as the Bawana seat albeit with different players - was wrested by the ruling Telugu Desam Party (TDP) of chief minister N Chandrababu Naidu, drawing the curtains on a shrill election campaign that saw him and his principal rival - YSR Congress chief YS Jagan Mohan Reddy trade sharp exchanges.