North and South Korea exchanged artillery fire over their maritime border on Monday, news agencies are reporting. Though no shells fell on land, South Korea as a precaution ordered residents of five islands to take cover in bomb shelters.

The incident began when North Korea fired shells south of the disputed sea border as part of a previously announced live-fire drill, leading South Korea to return fire, according to the Associated Press which quoted an official with South Korea’s Joint Chiefs of Staff. Reuters quoted defense officials in Seoul who said that South Korea also scrambled F-15s on its side of the maritime border.

“Some (North Korean) artillery fire landed in (the) southern part of Northern Limit Line — but in the water,” a South Korean Ministry of Defense spokesman told CNN. “We counter-fired over the Northern Limit Line”.

Asked what South Korea fired back at, the spokesman said, “We are not shooting at North Korea, just shooting into the sea.”

Lunch trays for students are left on tables of a cafeteria after students were evacuated to a shelter at a school on Yeonpyeong island, South Korea, near the West Sea border with North Korea, Monday, March 31, 2014. North and South Korea fired artillery shells into each other's waters Monday, a flare-up of animosity between the rivals that forced residents of five front-line South Korean islands to evacuate to shelters, South Korean officials said. (AP Photo)

Lunch trays for students are left on tables of a cafeteria after students were evacuated to a shelter at a school on Yeonpyeong island, South Korea, near the West Sea border with North Korea, Monday, March 31, 2014. North and South Korea fired artillery shells into each other’s waters Monday, a flare-up of animosity between the rivals that forced residents of five front-line South Korean islands to evacuate to shelters, South Korean officials said. (AP Photo)

North Korea’s advance announcement of its live-fire drill is considered to be unusual, as the reclusive state rarely signals its planning.

Reuters reported that North Korea accused Seoul of “gangster-like” behavior by “abducting” one of its fishing boats, threatening to strike back.

South Korea said it let the boat go after it had drifted into its side of the maritime border.

Though Monday’s incident appeared to end quickly and without injury, the AP noted that “there is worry in Seoul that increasing North Korean dissatisfaction could prompt a repeat of the weeks-long barrage of near-daily war rhetoric last spring that saw tensions soar as Pyongyang threatened nuclear strikes on Washington and Seoul following international condemnation of its third nuclear test.”

South Korean marine LVT-7 landing craft sail to shores through smoke screens during the U.S.-South Korea joint military exercises called Ssangyong, part of the Foal Eagle military exercises, in Pohang, South Korea, Monday, March 31, 2014. South Korea said North Korea has announced plans to conduct live-fire drills near the rivals' disputed western sea boundary. The planned drills Monday come after an increase in threatening rhetoric from Pyongyang and a series of rocket and ballistic missile launches in an apparent protest against the annual military exercises by Seoul and Washington. (AP Photo/Ahn Young-joon)

South Korean marine LVT-7 landing craft sail to shores through smoke screens during the U.S.-South Korea joint military exercises called Ssangyong, part of the Foal Eagle military exercises, in Pohang, South Korea, Monday, March 31, 2014. South Korea said North Korea has announced plans to conduct live-fire drills near the rivals’ disputed western sea boundary. The planned drills Monday come after an increase in threatening rhetoric from Pyongyang and a series of rocket and ballistic missile launches in an apparent protest against the annual military exercises by Seoul and Washington. (AP Photo/Ahn Young-joon)

For weeks now, Pyongyang has been upping its belligerent rhetoric and conducting rocket and missile launches that are believed to be a response to the annual U.S.-South Korean military drill “Foal Eagle” currently underway until mid-April.

On Sunday, North Korea threatened to carry out a fourth nuclear test, though according to the AP, South Korean officials say they don’t see signs of an imminent test.

CNN quoted a North Korean foreign ministry statement issued to the state-run KCNA news agency which said, “(We) would not rule out a new form of a nuclear test aimed at strengthening our nuclear deterrence.”

“The U.S. had better ponder over this and stop acting rashly,” the North Korean ministry added.

The AP quoted South Korean Defense Ministry deputy spokesman Wee Yong-sub saying Pyongyang’s early notice about Monday’s live-fire exercise was a “hostile” attempt to heighten tension between the two countries.