How to implement 3-way conference call movie talk with WebRTC Native Code for Android? Stack Overflow
Без кейворда
I’m attempting to implement 3-way movie talk inwards an Android app using the WebRTC Native Code package for Android (i.e. not using a WebView). I’ve written a signalling server using knot.js and used the Gottox socket.io java client library inwards the client app to connect to the server, exchange SDP packets and establish a 2-way movie talk connection.
However now I’m having problems going beyond that to a 3-way call. The AppRTCDemo app that comes with the WebRTC native code package demonstrates 2-way calls only (if a 3rd party attempts to join a room a "room total" message is returned).
According to this response (which doesn’t relate to Android specifically), I’m supposed to do it by creating numerous PeerConnections, so each talk participant will connect to the two other participants.
However, when I create more than one PeerConnectionClient (a Java class which wraps a PeerConection, which is implemented on the native side in libjingle_peerconnection_so.so), there is an exception thrown from inwards the library resulting from a conflict with both of them attempting to access the camera:
This happens when initializing the local client even before attempting to establish a connection so it’s not related to knot.js, socket.io or any of the signalling server stuff.
How do I get numerous PeerConnections to share the camera so that I can send the same movie to more than one peer?
One idea I had was to implement some kind of singleton camera class to substitute VideoCapturerAndroid that could be collective inbetween numerous connections, but I’m not even sure that would work and I’d like to know if there is a way to do 3-way calls using the API before I commence hacking around inwards the library.
Is it possible and if so, how?
I attempted sharing a VideoCapturerAndroid object inbetween numerous PeerConnectionClients, creating it for the very first connection only and passing it into the initialization function for the subsequent ones, but that resulted in this "Capturer can only be taken once!" exception when creating a 2nd VideoTrack from the VideoCapturer object for the 2nd peer connection:
Attempting to share the VideoTrack object inbetween PeerConnectionClients resulted in this error from the native code:
Sharing the MediaStream inbetween PeerConnectionClients results in the app abruptly closing, without any error message appearing in the Logcat.
Two Answers
The problem you are having is that PeerConnectionClient is not a wrapper around PeerConnection it contains a PeerConnection.
I noticed this question wasn’t answered so I wished to see if I could help out a bit. I looked into the source code and PeerConnectionClient is very much hard coded for a single remote peer. You would need to create a collection of PeerConnection objects rather then this line:
If you look around a bit more you would notice that it gets a bit more complicated then that.
The mediaStream logic in createPeerConnectionInternal should only be done once and you need to share the stream inbetween your PeerConnection objects like this:
You can consult the WebRTC spec or take a look at this stackoverflow question to confirm that the PeerConnection type was designed to only treat one peer. It is also somewhat vaguely implied here.
So you only maintain one mediaStream object:
So again the main idea is one MediaStream object and as many PeerConnection objects as you have peers you want to connect to. So you will not be using numerous PeerConnectionClient objects, but rather modify the single PeerConnectionClient to encapsulate the multi-client treating. If you do want to go with a design of numerous PeerConnectionClient objects for whatever reason you would just have to abstract the media stream logic (and any support types that should only be created once) out of it.
You will also need to maintain numerous remote movie tracks rather then the existing one:
You would obviously only care to render the one local camera and create numerous renderers for the remote connections.
I hope this is enough information to get you back on track.
With the help of Matthew Sanders’ response I managed to get it working, so in this response I’m going to describe in more detail one way of adapting the sample code to support movie conference calling:
Most of the switches need to be made in PeerConnectionClient , but also in the class that uses PeerConnectionClient , which is where you communicate with the signalling server and set up the connections.
Inwards PeerConnectionClient , the following member variables need to be stored per-connection:
In my application I needed a maximum of three connections (for a 4-way talk), so I just stored an array of each, but you could put them all inwards an object and have an array of objects.
I added a connectionId field to the PCObserver and SDPObserver classes, and inwards the PeerConnectionClient constructor I allocated the observer objects in the array and set the connectionId field for each observer object to its index in the array. All the methods of PCObserver and SDPObserver that reference the member variables listed above should be switched to index into the suitable array using the connectionId field.
The PeerConnectionClient callbacks need to be switched:
And also the following PeerConnectionClient methods:
As with the methods in the observer classes, all these functions need to be switched to use the connectionId to index into the suitable array of per-connection objects, instead of referencing the single objects they were previously. Any invocations of callback functions need to also be switched to pass the connectionId back.
I substituted createPeerConnection with a fresh function called createMultiPeerConnection , which is passed an array of VideoRenderer.Callbacks objects for displaying the remote movie stream, instead of a single one. The function calls createMediaConstraintsInternal() once and createPeerConnectionInternal() for each of the PeerConnection s, looping from zero to MAX_CONNECTIONS – one . The mediaStream object is created only on the very first call to createPeerConnectionInternal() , just by wrapping the initialization code in an if(mediaStream == null) check.
One complication I encountered was when when the app shuts down and the PeerConnection instances are closed and the MediaStream disposed of. In the sample code the mediaStream is added to a PeerConnection using addStream(mediaStream) , but the corresponding removeStream(mediaStream) function is never called ( dispose() is called instead). However this creates problems (a ref count assert in MediaStreamInterface in the native code) when there are more than one PeerConnection sharing a MediaStream object because dispose() finalizes the MediaStream , which should only happen when the last PeerConnection is closed. Calling removeStream() and close() is not enough either, because it doesn’t fully shut down the PeerConnection and this leads to an assert crash when disposing the PeerConnectionFactory object. The only fix I could find was to add the following code to the PeerConnection class:
And then calling these functions when finalizing each PeerConnection except the last:
How to implement 3-way conference call movie talk with WebRTC Native Code for Android? Stack Overflow
Без кейворда
I’m attempting to implement 3-way movie talk inwards an Android app using the WebRTC Native Code package for Android (i.e. not using a WebView). I’ve written a signalling server using knot.js and used the Gottox socket.io java client library inwards the client app to connect to the server, exchange SDP packets and establish a 2-way movie talk connection.
However now I’m having problems going beyond that to a 3-way call. The AppRTCDemo app that comes with the WebRTC native code package demonstrates 2-way calls only (if a 3rd party attempts to join a room a "room utter" message is returned).
According to this response (which doesn’t relate to Android specifically), I’m supposed to do it by creating numerous PeerConnections, so each talk participant will connect to the two other participants.
However, when I create more than one PeerConnectionClient (a Java class which wraps a PeerConection, which is implemented on the native side in libjingle_peerconnection_so.so), there is an exception thrown from inwards the library resulting from a conflict with both of them attempting to access the camera:
This happens when initializing the local client even before attempting to establish a connection so it’s not related to knot.js, socket.io or any of the signalling server stuff.
How do I get numerous PeerConnections to share the camera so that I can send the same movie to more than one peer?
One idea I had was to implement some kind of singleton camera class to substitute VideoCapturerAndroid that could be collective inbetween numerous connections, but I’m not even sure that would work and I’d like to know if there is a way to do 3-way calls using the API before I commence hacking around inwards the library.
Is it possible and if so, how?
I attempted sharing a VideoCapturerAndroid object inbetween numerous PeerConnectionClients, creating it for the very first connection only and passing it into the initialization function for the subsequent ones, but that resulted in this "Capturer can only be taken once!" exception when creating a 2nd VideoTrack from the VideoCapturer object for the 2nd peer connection:
Attempting to share the VideoTrack object inbetween PeerConnectionClients resulted in this error from the native code:
Sharing the MediaStream inbetween PeerConnectionClients results in the app abruptly closing, without any error message appearing in the Logcat.
Two Answers
The problem you are having is that PeerConnectionClient is not a wrapper around PeerConnection it contains a PeerConnection.
I noticed this question wasn’t answered so I dreamed to see if I could help out a bit. I looked into the source code and PeerConnectionClient is very much hard coded for a single remote peer. You would need to create a collection of PeerConnection objects rather then this line:
If you look around a bit more you would notice that it gets a bit more complicated then that.
The mediaStream logic in createPeerConnectionInternal should only be done once and you need to share the stream inbetween your PeerConnection objects like this:
You can consult the WebRTC spec or take a look at this stackoverflow question to confirm that the PeerConnection type was designed to only treat one peer. It is also somewhat vaguely implied here.
So you only maintain one mediaStream object:
So again the main idea is one MediaStream object and as many PeerConnection objects as you have peers you want to connect to. So you will not be using numerous PeerConnectionClient objects, but rather modify the single PeerConnectionClient to encapsulate the multi-client treating. If you do want to go with a design of numerous PeerConnectionClient objects for whatever reason you would just have to abstract the media stream logic (and any support types that should only be created once) out of it.
You will also need to maintain numerous remote movie tracks rather then the existing one:
You would obviously only care to render the one local camera and create numerous renderers for the remote connections.
I hope this is enough information to get you back on track.
With the help of Matthew Sanders’ response I managed to get it working, so in this reaction I’m going to describe in more detail one way of adapting the sample code to support movie conference calling:
Most of the switches need to be made in PeerConnectionClient , but also in the class that uses PeerConnectionClient , which is where you communicate with the signalling server and set up the connections.
Inwards PeerConnectionClient , the following member variables need to be stored per-connection:
In my application I needed a maximum of three connections (for a 4-way talk), so I just stored an array of each, but you could put them all inwards an object and have an array of objects.
I added a connectionId field to the PCObserver and SDPObserver classes, and inwards the PeerConnectionClient constructor I allocated the observer objects in the array and set the connectionId field for each observer object to its index in the array. All the methods of PCObserver and SDPObserver that reference the member variables listed above should be switched to index into the adequate array using the connectionId field.
The PeerConnectionClient callbacks need to be switched:
And also the following PeerConnectionClient methods:
As with the methods in the observer classes, all these functions need to be switched to use the connectionId to index into the adequate array of per-connection objects, instead of referencing the single objects they were previously. Any invocations of callback functions need to also be switched to pass the connectionId back.
I substituted createPeerConnection with a fresh function called createMultiPeerConnection , which is passed an array of VideoRenderer.Callbacks objects for displaying the remote movie stream, instead of a single one. The function calls createMediaConstraintsInternal() once and createPeerConnectionInternal() for each of the PeerConnection s, looping from zero to MAX_CONNECTIONS – one . The mediaStream object is created only on the very first call to createPeerConnectionInternal() , just by wrapping the initialization code in an if(mediaStream == null) check.
One complication I encountered was when when the app shuts down and the PeerConnection instances are closed and the MediaStream disposed of. In the sample code the mediaStream is added to a PeerConnection using addStream(mediaStream) , but the corresponding removeStream(mediaStream) function is never called ( dispose() is called instead). However this creates problems (a ref count assert in MediaStreamInterface in the native code) when there are more than one PeerConnection sharing a MediaStream object because dispose() finalizes the MediaStream , which should only happen when the last PeerConnection is closed. Calling removeStream() and close() is not enough either, because it doesn’t fully shut down the PeerConnection and this leads to an assert crash when disposing the PeerConnectionFactory object. The only fix I could find was to add the following code to the PeerConnection class:
And then calling these functions when finalizing each PeerConnection except the last:
How to implement 3-way conference call movie talk with WebRTC Native Code for Android? Stack Overflow
Без кейворда
I’m attempting to implement 3-way movie talk inwards an Android app using the WebRTC Native Code package for Android (i.e. not using a WebView). I’ve written a signalling server using knot.js and used the Gottox socket.io java client library inwards the client app to connect to the server, exchange SDP packets and establish a 2-way movie talk connection.
However now I’m having problems going beyond that to a 3-way call. The AppRTCDemo app that comes with the WebRTC native code package demonstrates 2-way calls only (if a 3rd party attempts to join a room a "room total" message is returned).
According to this response (which doesn’t relate to Android specifically), I’m supposed to do it by creating numerous PeerConnections, so each talk participant will connect to the two other participants.
However, when I create more than one PeerConnectionClient (a Java class which wraps a PeerConection, which is implemented on the native side in libjingle_peerconnection_so.so), there is an exception thrown from inwards the library resulting from a conflict with both of them attempting to access the camera:
This happens when initializing the local client even before attempting to establish a connection so it’s not related to knot.js, socket.io or any of the signalling server stuff.
How do I get numerous PeerConnections to share the camera so that I can send the same movie to more than one peer?
One idea I had was to implement some kind of singleton camera class to substitute VideoCapturerAndroid that could be collective inbetween numerous connections, but I’m not even sure that would work and I’d like to know if there is a way to do 3-way calls using the API before I embark hacking around inwards the library.
Is it possible and if so, how?
I attempted sharing a VideoCapturerAndroid object inbetween numerous PeerConnectionClients, creating it for the very first connection only and passing it into the initialization function for the subsequent ones, but that resulted in this "Capturer can only be taken once!" exception when creating a 2nd VideoTrack from the VideoCapturer object for the 2nd peer connection:
Attempting to share the VideoTrack object inbetween PeerConnectionClients resulted in this error from the native code:
Sharing the MediaStream inbetween PeerConnectionClients results in the app abruptly closing, without any error message appearing in the Logcat.
Two Answers
The problem you are having is that PeerConnectionClient is not a wrapper around PeerConnection it contains a PeerConnection.
I noticed this question wasn’t answered so I wished to see if I could help out a bit. I looked into the source code and PeerConnectionClient is very much hard coded for a single remote peer. You would need to create a collection of PeerConnection objects rather then this line:
If you look around a bit more you would notice that it gets a bit more complicated then that.
The mediaStream logic in createPeerConnectionInternal should only be done once and you need to share the stream inbetween your PeerConnection objects like this:
You can consult the WebRTC spec or take a look at this stackoverflow question to confirm that the PeerConnection type was designed to only treat one peer. It is also somewhat vaguely implied here.
So you only maintain one mediaStream object:
So again the main idea is one MediaStream object and as many PeerConnection objects as you have peers you want to connect to. So you will not be using numerous PeerConnectionClient objects, but rather modify the single PeerConnectionClient to encapsulate the multi-client treating. If you do want to go with a design of numerous PeerConnectionClient objects for whatever reason you would just have to abstract the media stream logic (and any support types that should only be created once) out of it.
You will also need to maintain numerous remote movie tracks rather then the existing one:
You would obviously only care to render the one local camera and create numerous renderers for the remote connections.
I hope this is enough information to get you back on track.
With the help of Matthew Sanders’ response I managed to get it working, so in this response I’m going to describe in more detail one way of adapting the sample code to support movie conference calling:
Most of the switches need to be made in PeerConnectionClient , but also in the class that uses PeerConnectionClient , which is where you communicate with the signalling server and set up the connections.
Inwards PeerConnectionClient , the following member variables need to be stored per-connection:
In my application I needed a maximum of three connections (for a 4-way talk), so I just stored an array of each, but you could put them all inwards an object and have an array of objects.
I added a connectionId field to the PCObserver and SDPObserver classes, and inwards the PeerConnectionClient constructor I allocated the observer objects in the array and set the connectionId field for each observer object to its index in the array. All the methods of PCObserver and SDPObserver that reference the member variables listed above should be switched to index into the suitable array using the connectionId field.
The PeerConnectionClient callbacks need to be switched:
And also the following PeerConnectionClient methods:
As with the methods in the observer classes, all these functions need to be switched to use the connectionId to index into the adequate array of per-connection objects, instead of referencing the single objects they were previously. Any invocations of callback functions need to also be switched to pass the connectionId back.
I substituted createPeerConnection with a fresh function called createMultiPeerConnection , which is passed an array of VideoRenderer.Callbacks objects for displaying the remote movie stream, instead of a single one. The function calls createMediaConstraintsInternal() once and createPeerConnectionInternal() for each of the PeerConnection s, looping from zero to MAX_CONNECTIONS – one . The mediaStream object is created only on the very first call to createPeerConnectionInternal() , just by wrapping the initialization code in an if(mediaStream == null) check.
One complication I encountered was when when the app shuts down and the PeerConnection instances are closed and the MediaStream disposed of. In the sample code the mediaStream is added to a PeerConnection using addStream(mediaStream) , but the corresponding removeStream(mediaStream) function is never called ( dispose() is called instead). However this creates problems (a ref count assert in MediaStreamInterface in the native code) when there are more than one PeerConnection sharing a MediaStream object because dispose() finalizes the MediaStream , which should only happen when the last PeerConnection is closed. Calling removeStream() and close() is not enough either, because it doesn’t fully shut down the PeerConnection and this leads to an assert crash when disposing the PeerConnectionFactory object. The only fix I could find was to add the following code to the PeerConnection class:
And then calling these functions when finalizing each PeerConnection except the last:
How to implement 3-way conference call movie talk with WebRTC Native Code for Android? Stack Overflow
Без кейворда
I’m attempting to implement 3-way movie talk inwards an Android app using the WebRTC Native Code package for Android (i.e. not using a WebView). I’ve written a signalling server using knot.js and used the Gottox socket.io java client library inwards the client app to connect to the server, exchange SDP packets and establish a 2-way movie talk connection.
However now I’m having problems going beyond that to a 3-way call. The AppRTCDemo app that comes with the WebRTC native code package demonstrates 2-way calls only (if a 3rd party attempts to join a room a "room total" message is returned).
According to this reaction (which doesn’t relate to Android specifically), I’m supposed to do it by creating numerous PeerConnections, so each talk participant will connect to the two other participants.
However, when I create more than one PeerConnectionClient (a Java class which wraps a PeerConection, which is implemented on the native side in libjingle_peerconnection_so.so), there is an exception thrown from inwards the library resulting from a conflict with both of them attempting to access the camera:
This happens when initializing the local client even before attempting to establish a connection so it’s not related to knot.js, socket.io or any of the signalling server stuff.
How do I get numerous PeerConnections to share the camera so that I can send the same movie to more than one peer?
One idea I had was to implement some kind of singleton camera class to substitute VideoCapturerAndroid that could be collective inbetween numerous connections, but I’m not even sure that would work and I’d like to know if there is a way to do 3-way calls using the API before I commence hacking around inwards the library.
Is it possible and if so, how?
I attempted sharing a VideoCapturerAndroid object inbetween numerous PeerConnectionClients, creating it for the very first connection only and passing it into the initialization function for the subsequent ones, but that resulted in this "Capturer can only be taken once!" exception when creating a 2nd VideoTrack from the VideoCapturer object for the 2nd peer connection:
Attempting to share the VideoTrack object inbetween PeerConnectionClients resulted in this error from the native code:
Sharing the MediaStream inbetween PeerConnectionClients results in the app abruptly closing, without any error message appearing in the Logcat.
Two Answers
The problem you are having is that PeerConnectionClient is not a wrapper around PeerConnection it contains a PeerConnection.
I noticed this question wasn’t answered so I dreamed to see if I could help out a bit. I looked into the source code and PeerConnectionClient is very much hard coded for a single remote peer. You would need to create a collection of PeerConnection objects rather then this line:
If you look around a bit more you would notice that it gets a bit more complicated then that.
The mediaStream logic in createPeerConnectionInternal should only be done once and you need to share the stream inbetween your PeerConnection objects like this:
You can consult the WebRTC spec or take a look at this stackoverflow question to confirm that the PeerConnection type was designed to only treat one peer. It is also somewhat vaguely implied here.
So you only maintain one mediaStream object:
So again the main idea is one MediaStream object and as many PeerConnection objects as you have peers you want to connect to. So you will not be using numerous PeerConnectionClient objects, but rather modify the single PeerConnectionClient to encapsulate the multi-client treating. If you do want to go with a design of numerous PeerConnectionClient objects for whatever reason you would just have to abstract the media stream logic (and any support types that should only be created once) out of it.
You will also need to maintain numerous remote movie tracks rather then the existing one:
You would obviously only care to render the one local camera and create numerous renderers for the remote connections.
I hope this is enough information to get you back on track.
With the help of Matthew Sanders’ reaction I managed to get it working, so in this response I’m going to describe in more detail one way of adapting the sample code to support movie conference calling:
Most of the switches need to be made in PeerConnectionClient , but also in the class that uses PeerConnectionClient , which is where you communicate with the signalling server and set up the connections.
Inwards PeerConnectionClient , the following member variables need to be stored per-connection:
In my application I needed a maximum of three connections (for a 4-way talk), so I just stored an array of each, but you could put them all inwards an object and have an array of objects.
I added a connectionId field to the PCObserver and SDPObserver classes, and inwards the PeerConnectionClient constructor I allocated the observer objects in the array and set the connectionId field for each observer object to its index in the array. All the methods of PCObserver and SDPObserver that reference the member variables listed above should be switched to index into the adequate array using the connectionId field.
The PeerConnectionClient callbacks need to be switched:
And also the following PeerConnectionClient methods:
As with the methods in the observer classes, all these functions need to be switched to use the connectionId to index into the suitable array of per-connection objects, instead of referencing the single objects they were previously. Any invocations of callback functions need to also be switched to pass the connectionId back.
I substituted createPeerConnection with a fresh function called createMultiPeerConnection , which is passed an array of VideoRenderer.Callbacks objects for displaying the remote movie stream, instead of a single one. The function calls createMediaConstraintsInternal() once and createPeerConnectionInternal() for each of the PeerConnection s, looping from zero to MAX_CONNECTIONS – one . The mediaStream object is created only on the very first call to createPeerConnectionInternal() , just by wrapping the initialization code in an if(mediaStream == null) check.
One complication I encountered was when when the app shuts down and the PeerConnection instances are closed and the MediaStream disposed of. In the sample code the mediaStream is added to a PeerConnection using addStream(mediaStream) , but the corresponding removeStream(mediaStream) function is never called ( dispose() is called instead). However this creates problems (a ref count assert in MediaStreamInterface in the native code) when there are more than one PeerConnection sharing a MediaStream object because dispose() finalizes the MediaStream , which should only happen when the last PeerConnection is closed. Calling removeStream() and close() is not enough either, because it doesn’t fully shut down the PeerConnection and this leads to an assert crash when disposing the PeerConnectionFactory object. The only fix I could find was to add the following code to the PeerConnection class:
And then calling these functions when finalizing each PeerConnection except the last: