There is multiple way to achieve what you want to do, you can :
- rasterize your vector layer and set your border's pixel with an aberrant value, then use the raster calculator to obtain a raster without your border.
- keep your borders as a vector layer and use the print composer to get a png file with your border on top of your raster.
I only have two layers in my project for the 2 script I propose, this is what my project looks like :

This is a script for the first solution, the formula for the raster calculator is based on the first answer of this post How to set all pixels with value <= 0 to “nodata” in DEM raster? :
vector_layer = QgsProject.instance().mapLayersByName("border")[0]
raster_layer = QgsProject.instance().mapLayersByName("dtm")[0]
output_path = "/path/of/the/output/output_name.png"
Transform your borders into a raster, the value of the pixel is 9999
burn_value = 9999
rasterize = processing.run("gdal:rasterize", {'INPUT':vector_layer,'FIELD':'','BURN':burn_value,'UNITS':0,'WIDTH':raster_layer.width(),'HEIGHT':raster_layer.height(),'EXTENT':raster_layer.extent(),'NODATA':0,'OPTIONS':'','DATA_TYPE':5,'INIT':None,'INVERT':False,'EXTRA':'','OUTPUT':'TEMPORARY_OUTPUT'})
border_raster = rasterize['OUTPUT']
#iface.addRasterLayer(border_raster)
Add a value, 0, to the noData cells of your border raster
fillnodata = processing.run("native:fillnodata", {'INPUT':border_raster,'BAND':1,'FILL_VALUE':0,'OUTPUT':'TEMPORARY_OUTPUT'})
border_raster = fillnodata['OUTPUT']
#iface.addRasterLayer(border_raster)
Soustract your border raster to your original raster, you'll get value under 0 when a pixel is on a border
Then put all the values below 0 to NoData
formula = '(((A-B)>0)(A-B)) / (((A-B)>0)1 + ((A-B)<=0)*0)'
rastercalculator = processing.run("gdal:rastercalculator", {'INPUT_A':raster_layer,'BAND_A':1,'INPUT_B':border_raster,'BAND_B':1,'INPUT_C':None,'BAND_C':None,'INPUT_D':None,'BAND_D':None,'INPUT_E':None,'BAND_E':None,'INPUT_F':None,'BAND_F':None,'FORMULA':formula,'NO_DATA':None,'RTYPE':5,'OPTIONS':'','EXTRA':'','OUTPUT':'TEMPORARY_OUTPUT'})
result_raster = rastercalculator['OUTPUT']
iface.addRasterLayer(result_raster)
Save your final raster as a png
processing.run("grass7:r.out.png", {'input':result_raster,'compression':0,'-t':False,'-w':False,'output':output_path,'GRASS_REGION_PARAMETER':None,'GRASS_REGION_CELLSIZE_PARAMETER':0})
This is the result, the problem is that you cannot change the style of the raster.

Here is a script for the second solution, maybe you'll need to tweak the value of rect_scale, I put it on 1.4 for my layers. :
def create_png(layers, path, name):
project = QgsProject.instance()
canvas = iface.mapCanvas()
manager = project.layoutManager()
layoutName = 'Layout1'
layouts_list = manager.printLayouts()
# remove any duplicate layouts
for layout in layouts_list:
if layout.name() == layoutName:
manager.removeLayout(layout)
layout = QgsPrintLayout(project)
layout.initializeDefaults()
layout.setName(layoutName)
manager.addLayout(layout)
# create map item in the layout
map = QgsLayoutItemMap(layout)
map.setRect(20, 20, 20, 20)
# set the map extent
ms = QgsMapSettings()
context = QgsRenderContext.fromMapSettings(ms)
ms.setLayers(layers) # set layers to be mapped
rect = QgsRectangle(ms.fullExtent())
rect.scale(1.4)
ms.setExtent(rect)
map.setExtent(rect)
map.setBackgroundColor(QColor(255, 255, 255, 0))
layout.addLayoutItem(map)
map.attemptMove(QgsLayoutPoint(0, 0, QgsUnitTypes.LayoutMillimeters))
map.attemptResize(QgsLayoutSize(297, 210, QgsUnitTypes.LayoutMillimeters))
layout = manager.layoutByName(layoutName)
exporter = QgsLayoutExporter(layout)
fn = path + name
exporter.exportToImage(fn + '.png', QgsLayoutExporter.ImageExportSettings())
vector_layer = QgsProject.instance().mapLayersByName("border")[0]
raster_layer = QgsProject.instance().mapLayersByName("dtm")[0]
layers = [vector_layer, raster_layer]
name = "name_of_the_output"
path = "/path/to/output/png/"
create_png(layers, path, name)
This is the result of the second solution :

You can also mix both solution and create one raster with your border as pixel and then export it as a map with the code of the second solution. You can keep the style you want for your layer.