Multi-View Signal Codec

{
  "publication_number": "US2013147915A1",
  "country": "US",
  "kind": "A1",
  "title": "Multi-View Signal Codec",
  "abstract": "Embodiments are described which exploit a finding, according to which a higher compression rate or better rate/distortion ratio may be achieved by adopting or predicting second coding parameters used for encoding a second view of the multi-view signal from first coding parameters used in encoding a first view of the multi-view signal. In other words, the inventors found out that the redundancies between views of a multi-view signal are not restricted to the views themselves, such as the video information thereof, but that the coding parameters in parallely encoding these views show similarities which may be exploited in order to further improve the coding rate.",
  "claims": [
    "1. A decoder configured to: reconstruct a first view of a multi-view signal from a data stream by, according to first coding parameters acquired from the data stream, predicting a current portion of the first view from a first previously reconstructed portion of the multi-view signal, reconstructed from the data stream by the decoder prior to the reconstruction of the current portion of the first view and correcting a prediction error of the prediction of the current portion of the first view using first correction data comprised in the data stream; at least partially adopt or predict second coding parameters from the first coding parameters; and reconstruct a second view of the multi-view signal from the data stream by, according to the second coding parameters, predicting a current portion of the second view from a second previously reconstructed portion of the multi-view signal, reconstructed from the data stream by the decoder prior to the reconstruction of the current portion of the second view and correcting a prediction error of the prediction of the current portion of the second view using second correction data comprised in the data stream. 2. The decoder according to claim 1, wherein the decoder is further configured to extract an intermediate view from the first and second views. 3. The decoder according to claim 1, wherein each of the first view and the second view comprises a video captured from a respective camera position, and associated depth/disparity map data. 4. The decoder according to claim 3, wherein the decoder is configured to: reconstruct the video of the first view from the data stream by, according to a first portion of the first coding parameters, predicting a current portion of the video of the first view from a third previously reconstructed portion of the multi-view signal, reconstructed from the data stream by the decoder prior to the reconstruction of the current portion of the video of the first view and correcting a prediction error of the prediction of the current portion of the video of the first view using a first subset of the first correction data comprised in the data stream, at least partially adopt or predict a second portion of the first coding parameters from the first portion of the first coding parameters, reconstruct the depth/disparity map data of the first view from the data stream by, according to the second portion of the first coding parameters, predicting a current portion of the depth/disparity map data of the first view from a fourth previously reconstructed portion of the multi-view signal, reconstructed from the data stream by the decoder prior to the reconstruction of the current portion of the depth/disparity map data of the first view and correcting a prediction error of the prediction of the current portion of the depth/disparity map data of the first view using a second subset of the first correction data. 5. The decoder according to claim 4, wherein the decoder is configured to: predict the current portion of the video of the first view from a first previously reconstructed portion of the video of the first view, reconstructed from the data stream by the decoder prior to the reconstruction of the current portion of the video of the first view, and predict the current portion of the depth/disparity map data of the first view from a first previously reconstructed portion of the depth/disparity map data of the first view, reconstructed from the data stream by the decoder prior to the reconstruction of the current portion of the depth/disparity map data of the first view. 6. The decoder according to claim 3, wherein the decoder is configured to: at least partially adopt or predict a first portion of the second coding parameters from the first coding parameters, reconstruct the video of the second view from the data stream by, according to the first portion of the second coding parameters, predicting a current portion of the video of the second view from a fifth previously reconstructed portion of the multi-view signal, reconstructed from the data stream by the decoder prior to the reconstruction of the current portion of the video of the second view and correcting a prediction error of the prediction of the current portion of the video of the second view using a first subset of the second correction data comprised in the data stream, at least partially adopt or predict a second portion of the second coding parameters from the first coding parameters and/or the first portion of the second coding parameters, and reconstruct the depth/disparity map data of the second view from the data stream by, according to the second portion of the second coding parameters, predicting a current portion of the depth/disparity map data of the second view from a sixth previously reconstructed portion of the multi-view signal, reconstructed from the data stream by the decoder prior to the reconstruction of the current portion of the depth/disparity map data of the second view and correcting a prediction error of the prediction of the current portion of the depth/disparity map data of the second view using a second subset of the second correction data. 7. The decoder according to claim 6, wherein the decoder is configured to: predict the current portion of the video of the second view from a previously reconstructed portion of the video of the second view, reconstructed from the data stream by the decoder prior to the reconstruction of the current portion of the video of the second view, or from a second previously reconstructed portion of the video of the first view, reconstructed from the data stream by the decoder prior to the reconstruction of the current portion of the video of the second view, and predict the current portion of the depth/disparity map data of the second view from a previously reconstructed portion of the depth/disparity map data of the second view, reconstructed from the data stream by the decoder prior to the reconstruction of the current portion of the depth/disparity map data of the second view, or from a second previously reconstructed portion of the depth/disparity map data of the first view, reconstructed from the data stream by the decoder prior to the reconstruction of the current portion of the depth/disparity map data of the second view. 8. The decoder according to claim 6, wherein the decoder is configured to: detect an edge in the a current frame of the video of the second view and determine a wedgelet separation line so as to extend along the edge, and in reconstructing the depth/disparity map data of the second view from the data stream, set a border of the current portion of a depth/disparity map of the depth/disparity map data of the second view, associated with the current frame of the video the second view, so as to coincide with the wedgelet separation line. 9. The decoder according to claim 8, wherein the decoder is configured to perform the prediction within the reconstructing the depth/disparity map data of the second view from the data stream segment-wise in units of segments to which the current portion belongs, with using distinct sets of prediction parameters for the segments. 10. The decoder according to claim 8, wherein the decoder is configured such that the wedgelet separation line is a straight line and the decoder is configured to divide a block of a pre-segmentation of the depth/disparity map data of the second view along the wedgelet separation line so that the two neighboring segments are wedgelet-shaped segments together forming the block of the pre-segmentation. 11. The decoder according to claim 1, wherein the decoder is configured such that the first and second coding parameters are first and second prediction parameters, respectively, controlling the prediction of the current portion of the first view and the prediction of the current portion of the second view, respectively. 12. The decoder according to claim 1, wherein the current portions are segments of a segmentation of frames of the video of the first and second view. respectively. 13. The decoder according to claim 1, wherein the decoder is configured to reconstruct the first view of the multi-view signal from the data-stream by performing the predicting and correcting of the current portion thereof at a first spatial resolution, and to reconstruct the second view of the multi-view signal from the data-stream by performing the predicting and correcting of the current portion thereof at a second spatial resolution lower than the first spatial resolution, with then up-sampling the reconstructed current portion of the second view from the second spatial resolution to the first spatial resolution. 14. The decoder according to claim 13, wherein the decoder is configured to, in at least partially adopt or predict the second coding parameters from the first coding parameters, scaling the first coding parameters according to a ratio between the first and second spatial resolutions. 15. An encoder configured to: encode a first view of a multi-view signal into a data stream by determining first coding parameters, according to the first coding parameters, predicting a current portion of the first view from a first previously encoded portion of the multi-view signal, encoded into the data stream by the encoder prior to the encoding of the current portion of the first view, and determining a prediction error of the prediction of the current portion of the first view in order to acquire first correction data, and inserting the first coding parameters and the first correction data into the data stream; and encode a second view of the multi-view signal into the data stream by, determining second coding parameters by adopting or predicting the second coding parameters from the first coding parameters; according to the second coding parameters, predicting a current portion of the second view from a second previously encoded portion of the multi-view signal, encoded into the data stream by the encoder prior to the encoding of the current portion of the second view and determining a prediction error of the prediction of the current portion of the second view in order to acquire second correction data comprised in the data stream, and inserting the second correction data into the data stream. 16. A data stream comprising: a first part into which a first view of a multi-view signal is encoded, the first part comprising first correction parameters and first coding parameters such that, according to the first coding parameters, a current portion of the first view is predictable from a first previously encoded portion of the multi-view signal, encoded into the data stream prior to the current portion of the first view, and a prediction error of the prediction of the current portion of the first view is correctable using the first correction data, and a second part into which a second view of the multi-view signal is encoded, the second part comprising second correction parameters such that according to second coding parameters predictable from, or adopted from the first coding parameters, a current portion of the second view is predictable from a second previously encoded portion of the multi-view signal, encoded into the data stream prior to the encoding of the current portion of the second view and a prediction error of the prediction of the current portion of the second view is correctable using the second correction data. 17. A decoding method comprising: reconstructing a first view of a multi-view signal from a data stream by, according to first coding parameters acquired from the data stream, predicting a current portion of the first view from a first previously reconstructed portion of the multi-view signal, reconstructed from the data stream by the decoder prior to the reconstruction of the current portion of the first view and correcting a prediction error of the prediction of the current portion of the first view using first correction data comprised in the data stream; at least partially adopting or predicting second coding parameters from the first coding parameters; and reconstructing a second view of the multi-view signal from the data stream by, according to the second coding parameters, predicting a current portion of the second view from a second previously reconstructed portion of the multi-view signal, reconstructed from the data stream by the decoder prior to the reconstruction of the current portion of the second view and correcting a prediction error of the prediction of the current portion of the second view using second correction data comprised in the data stream. 18. An encoding method comprising: encoding a first view of a multi-view signal into a data stream by determining first coding parameters, according to the first coding parameters, predicting a current portion of the first view from a first previously encoded portion of the multi-view signal, encoded into the data stream by the encoder prior to the encoding of the current portion of the first view, and determining a prediction error of the prediction of the current portion of the first view in order to acquire first correction data, and inserting the first coding parameters and the first correction data into the data stream; and encoding a second view of the multi-view signal into the data stream by determining second coding parameters by adopting or predicting the second coding parameters from the first coding parameters, according to the second coding parameters, predicting a current portion of the second view from a second previously encoded portion of the multi-view signal, encoded into the data stream by the encoder prior to the encoding of the current portion of the second view and determining a prediction error of the prediction of the current portion of the second view in order to acquire second correction data comprised in the data stream, and inserting the second correction data into the data stream. 19. A computer program comprising a program code for performing, when running on a computer, a decoding method comprising: reconstructing a first view of a multi-view signal from a data stream by, according to first coding parameters acquired from the data stream, predicting a current portion of the first view from a first previously reconstructed portion of the multi-view signal, reconstructed from the data stream by the decoder prior to the reconstruction of the current portion of the first view and correcting a prediction error of the prediction of the current portion of the first view using first correction data comprised in the data stream; at least partially adopting or predicting second coding parameters from the first coding parameters; and reconstructing a second view of the multi-view signal from the data stream by, according to the second coding parameters, predicting a current portion of the second view from a second previously reconstructed portion of the multi-view signal, reconstructed from the data stream by the decoder prior to the reconstruction of the current portion of the second view and correcting a prediction error of the prediction of the current portion of the second view using second correction data comprised in the data stream. 20. A computer program comprising a program code for performing, when running on a computer, an encoding method comprising: encoding a first view of a multi-view signal into a data stream by determining first coding parameters, according to the first coding parameters, predicting a current portion of the first view from a first previously encoded portion of the multi-view signal, encoded into the data stream by the encoder prior to the encoding of the current portion of the first view, and determining a prediction error of the prediction of the current portion of the first view in order to acquire first correction data, and inserting the first coding parameters and the first correction data into the data stream; and encoding a second view of the multi-view signal into the data stream by, determining second coding parameters by adopting or predicting the second coding parameters from the first coding parameters; according to the second coding parameters, predicting a current portion of the second view from a second previously encoded portion of the multi-view signal, encoded into the data stream by the encoder prior to the encoding of the current portion of the second view and determining a prediction error of the prediction of the current portion of the second view in order to acquire second correction data comprised in the data stream, inserting the second correction data into the data stream. 21. The decoder according to claim 4, wherein the first coding parameters define a segmentation of frames of the video of the first view, and the decoder is configured to, in reconstructing the depth/disparity map data of the first view, use the segmentation of the frames of the video of the first view as a pre-segmentation of the depth/disparity map data of the first view. 22. The decoder according to claim 4, wherein the decoder is configured to: use a reconstructed portion of the current frame of the video of the first view to predict a location of a wedgelet separation line, and in reconstructing the depth/disparity map data of the first view from the data stream, set a border of the current portion of a depth/disparity map of the depth/disparity map data of the first view, associated with the current frame of the video of the first view, so as to coincide with the wedgelet separation line. 23. The decoder according to claim 8, wherein the decoder is configured to: in reconstructing the depth/disparity map data of the first view from the data stream, use a first wedgelet separation line in the depth/disparity map data of the first view, use the first wedgelet separation line as a predictor for, or adopt the first wedgelet separation line as, a second wedgelet separation line in the depth/disparity map data of the second view, and in reconstructing the depth/disparity map data of the second view from the data stream, set a border of the current portion of a depth/disparity map of the depth/disparity map data of the second view, associated with the current frame of the video the second view, so as to coincide with the second wedgelet separation line."
  ],
  "cpc": [
    "H04N 19/597",
    "H04N 13/0048",
    "H04N 13/161",
    "H04N 19/139",
    "H04N 19/14",
    "H04N 19/147",
    "H04N 19/159",
    "H04N 19/176",
    "H04N 19/197",
    "H04N 19/463",
    "H04N 19/61",
    "H04N 19/88"
  ],
  "assignees": [
    "ANGEWANDTEN FORSCHUNG E V FRAUNHOFER GES ZUR FOERDERUNG DER",
    "FRAUNHOFER GES FORSCHUNG"
  ],
  "filing_date": "2013-02-08",
  "publication_date": "2013-06-13",
  "priority_date": "2010-08-11",
  "application_number": "US-201313762608-A",
  "family_id": "44504420",
  "citations": [
    "US2010142614A1",
    "US2010284466A1",
    "US2012114039A1",
    "US2012200669A1"
  ]
}