Segmentation error with avcodec_encode_video2

Question

I have some problems trying to encode an AVFrame into a package.

Before reading all the code, the input material works, I tested it. The output is given from an example here . I think there is a problem. But a segmentation error occurs in a loop near the end.

Here is my reduced code:

void nmain() { // input stuff AVFormatContext *formatCtxIn=0; AVInputFormat *formatIn=0; AVCodecContext *codecCtxIn=0; AVCodec *codecIn; AVPacket *pktIn; av_register_all(); avdevice_register_all(); avcodec_register_all(); formatIn = av_find_input_format("dshow"); if(!formatIn) return; AVDictionary *avoption=0; av_dict_set(&avoption, "rtbufsize", "1000000000", NULL); if(avformat_open_input(&formatCtxIn, "video=Integrated Camera", formatIn, &avoption)!=0) return; if(avformat_find_stream_info(formatCtxIn, NULL)<0) return; codecCtxIn = formatCtxIn->streams[0]->codec; codecIn = avcodec_find_decoder(codecCtxIn->codec_id); if(avcodec_open2(codecCtxIn, codecIn, NULL)<0) return; // end input stuff //------------------------------------------------------------------------------ // output stuff AVOutputFormat *formatOut=0; AVFormatContext *formatCtxOut=0; AVStream *streamOut=0; AVFrame *frame=0; AVCodec *codecOut=0; AVPacket *pktOut; const char *filename = "test.mpeg"; formatOut = av_guess_format(NULL, filename, NULL); if(!formatOut) formatOut = av_guess_format("mpeg", NULL, NULL); if(!formatOut) return; formatCtxOut = avformat_alloc_context(); if(!formatCtxOut) return; formatCtxOut->oformat = formatOut; sprintf(formatCtxOut->filename, "%s", filename); if(formatOut->video_codec != AV_CODEC_ID_NONE) { AVCodecContext *ctx; codecOut = avcodec_find_encoder(formatOut->video_codec); if(!codecOut) return; streamOut = avformat_new_stream(formatCtxOut, codecOut); if(!streamOut) return; ctx = streamOut->codec; ctx->bit_rate = 400000; ctx->width = 352; ctx->height = 288; ctx->time_base.den = 25; ctx->time_base.num = 1; ctx->gop_size = 12; ctx->pix_fmt = AV_PIX_FMT_YUV420P; if(ctx->codec_id == AV_CODEC_ID_MPEG2VIDEO) ctx->max_b_frames = 2; if(ctx->codec_id == AV_CODEC_ID_MPEG1VIDEO) ctx->mb_decision = 2; if(formatCtxOut->oformat->flags & AVFMT_GLOBALHEADER) ctx->flags |= CODEC_FLAG_GLOBAL_HEADER; } if(streamOut) { AVCodecContext *ctx; ctx = streamOut->codec; if(avcodec_open2(ctx, codecOut, NULL) < 0) return; } if(!(formatCtxOut->flags & AVFMT_NOFILE)) if(avio_open(&formatCtxOut->pb, filename, AVIO_FLAG_WRITE) < 0) return; avformat_write_header(formatCtxOut, NULL); // doit pktIn = new AVPacket; pktOut = new AVPacket; av_init_packet(pktOut); pktOut->data = 0; frame = avcodec_alloc_frame(); if(!frame) return; for(;;) { if(av_read_frame(formatCtxIn, pktIn) >= 0) { av_dup_packet(pktIn); int fff; if(avcodec_decode_video2(codecCtxIn, frame, &fff, pktIn) < 0) std::cout << "bad frame" << std::endl; if(!fff) return; // ok static int counter=0; SaveFrame(frame, codecCtxIn->width, codecCtxIn->height, counter++); // work fine // here a segmentation fault is occured. if(avcodec_encode_video2(streamOut->codec, pktOut, frame, &fff) < 0) std::cout << "bad frame" << std::endl; } } } // only for testing // add to ensure frame is valid void SaveFrame(AVFrame *pFrame, int width, int height, int iFrame) { FILE *pFile; char szFilename[32]; int y; // Open file sprintf(szFilename, "frame%d.ppm", iFrame); pFile=fopen(szFilename, "wb"); if(pFile==NULL) return; // Write header fprintf(pFile, "P6\n%d %d\n255\n", width, height); // Write pixel data for(y=0; y<height; y++) fwrite(pFrame->data[0]+y*pFrame->linesize[0], 1, width*3, pFile); // Close file fclose(pFile); } 

What am I doing wrong?

During debugging, I did not find any problems with the parameters. streamOut->codec populated. pktOut is highlighted and the frame populated with a pre-encoded image. I think the problem is creating the output codec, but watching the example and looking at doxypages seems right.

Route tracing is done from QT using msvc11 and framework 5.

I also tried to work with a doctor. memory and get the following:

 Error #26: UNADDRESSABLE ACCESS: reading 0x00000000-0x00000004 4 byte(s) # 0 replace_memcpy [d:\derek\drmemory\withwiki\trunk\drmemory\replace.c:203] # 1 avcodec-54.dll!ff_dct_quantize_c +0xd463 (0x6a482364 <avcodec-54.dll+0x3c2364>) # 2 avcodec-54.dll!avcodec_encode_video2+0xb7 (0x6a56a5b8 <avcodec-54.dll+0x4aa5b8>) # 3 nmain [d:\prg\tests\recording system-qt\libav\recsys\main.cpp:610] # 4 main [d:\prg\tests\recording system-qt\libav\recsys\main.cpp:182] Note: @0:00:06.318 in thread 5312 Note: instruction: mov (%edx) -> %ebx 

It seems that the reading process while memcpy is going wrong.

---

Version:

I forgot to mention the libav / ffmpeg version that I use:

 libavutil 51. 76.100 / 51. 76.100 libavcodec 54. 67.100 / 54. 67.100 libavformat 54. 33.100 / 54. 33.100 libavdevice 54. 3.100 / 54. 3.100 libavfilter 3. 19.103 / 3. 19.103 libswscale 2. 1.101 / 2. 1.101 libswresample 0. 16.100 / 0. 16.100 libpostproc 52. 1.100 / 52. 1.100 

---

Addendum:

The SafeFrame function SafeFrame copied from Tutorial 1 .

Answer 1

Finally, I solved my problem.

The problem (other than the libav documentation) of avpacket is not a (real) copy of the image in the package. it just points to the packet data. You should make a copy, or better, for you to enable libav.

So, first I created a new avframe for output and a buffer pointed to by the output avframe.

 AVFrame *outpic = avcodec_alloc_frame(); nbytes = avpicture_get_size(codecCtxOut->pix_fmt, codecCtxOut->width, codecCtxOut->height); uint8_t* outbuffer = (uint8_t*)av_malloc(nbytes); 

This buffer is used to convert from input to output. Then in the loop I have to fill outpic (avframe) with a buffer. I found in the code that this function fills the plane pointers with a buffer. see here

 avpicture_fill((AVPicture*)outpic, outbuffer, AV_PIX_FMT_YUV420P, codecCtxOut->width, codecCtxOut->height); 

Then I converted inpic to outpic using sws_scale . But first you need to configure swscontext.

 SwsContext* swsCtx_ = sws_getContext(codecCtxIn->width, codecCtxIn->height, codecCtxIn->pix_fmt, codecCtxOut->width, codecCtxOut->height, codecCtxOut->pix_fmt, SWS_BICUBIC, NULL, NULL, NULL); sws_scale(swsCtx_, inpic->data, inpic->linesize, 0, codecCtxIn->height, outpic->data, outpic->linesize); 

Then you can encode outpic to pktout (avpacket for output). But first release the output packet, otherwise you will get an error and a leak ... see here

 av_free_packet(pktOut); if(avcodec_encode_video2(streamOut->codec, pktOut, outpic, &fff) < 0) { std::cout << "shit frame" << std::endl; continue; } // and write it to the file formatOut->write_packet(formatCtxOut, pktOut); 

So now it works (almost perfectly) for me. There is still a small memory leak, but I can notice this later.

Answer 2

I see at least two problems with this recoding cycle:

1) You do not check if the decoder has created a frame. Many decoders have a delay between input and output, so a decoding call does not necessarily create a frame, even if an error does not occur. You just need to keep sending packets to the decoder until it starts returning frames (and then clears it with NULL packets at the end, as described in the documentation ).

As a result, you send the encoder an uninitialized frame, which is probably the cause of the failure.

2) Another problem that I see is that you initialize the output package only once. As the documentation says

The user can provide an output buffer by setting avpkt-> data and avpkt-> size before calling the function, but if the size of the data provided by the user is not large enough, encoding will fail. All other fields in AVPacket will be reset by the encoder using av_init_packet (). If avpkt-> data is NULL, the encoder will select it. The encoder will set the size avpkt-> to the size of the output packet. The returned data (if any) belongs to the caller, he is responsible for the release.

So, if you only initialize it once before starting the conversion cycle, at each iteration after the first it will contain the old data. The encoder will think that you want to use this buffer to encode and overwrite it. It will end in tears if you already passed this packet to the multiplexer or something like that. Therefore, before each encoding call, be sure to run the batch data and the size to NULL / 0.