i think i might have done what you're requesting though i did it in the service of a different thing.


oh great.


i have thirty minutes that i've more tightly transcribed with reference to individual channels.


okay.


okay that's great.


that's great too.


and i could and


yeah so


hopefully that's not the same meeting that we did.


and


no actually it's a different meeting.


good.


so um so the you know we have the they transcribe as if it's one channel with these with the slashes to separate the overlapping parts.


and then we run it through then it then i'm going to edit it and i'm going to run it through channelize which takes it into dave gelbart's format.


yeah.


and then you have all these things split across according to channel.


and then that means that if a person contributed more than once in a given overlap during that time bend that that two parts of the utterance end up together it's the same channel.


and then i took his tool and last night for the first thirty minutes of one of these transcripts i tightened up the um boundaries on individual speakers' channels.


because his his interface allows me to have total flexibility in the time tags across the channels.


and um so.


so current this week.


so yeah yeah that that that's great but what would be nice to have some more meetings not just one meeting to to be sure that that there is a system.


yes.


might not be what you need.


yeah so if we could get a couple meetings done with that level of precision i think that would be a good idea.


okay.


oh okay.


uh how how much


so the meetings vary in length.


what are we talking about in terms of the number of minutes you'd like to have as your as your training set?


it seems to me that it would be good to have a few minutes from from different meetings.


so


but i'm not sure about how much.


okay now you're saying different meetings because of different speakers or because of different audio quality or both or


both.


different different number of speakers different speakers different conditions.


yeah we don't have that much variety in meetings yet.


uh i mean we have this meeting and the feature meeting and we have a couple others that we have uh couple examples of but but uh


yeah


yeah.


uhhuh.


even probably with the gains differently will affect it you mean


uh not really as


potentially.


oh because you use the normalization?


because of the normalization yeah.


okay.


oh okay.


we can try running


we haven't done this yet because um uh andreas is is going to move over the s r i recognizer.


basically i ran out of machines at s r i.


because we're running the evals.


and i just don't have machine time there.


but once that's moved over uh hopefully in a a couple days then we can take um what jane just told us about as the presegmented the the segmentations that you did at level eight or at some threshold that jane right and try doing forced alignment um on the word strings.


oh shoot!


the presegment.


yeah.


with the recognizer?


and if it's good then that will that may give you a good boundary.


of course if it's good we don't then we're we're fine.


but i don't know yet whether these segments that contain a lot of pauses around the words will work or not.


i i would quite like to have some manually transcribed references for for the system as i'm not sure if if it's really good to compare with with some other automatic found boundaries.


right.


well no if we were to start with this and then tweak it manually would that that would be okay?


yeah sure.


they might be okay.


okay.


it you know it really depends on a lot of things.


but i would have maybe a transcriber uh look at the result of a forced alignment and then adjust those.


to adjust them or


yeah yeah.


that might save some time.


if they're horrible it won't help at all.


yeah great.


but they might not be horrible.


yeah.


so but i'll let you know when we uh have that.


okay great.


how many minutes would you want from


i mean we could easily get a section you know like say a minute or so from every meeting that we have so from the newer ones that we're working on everyone that we have.


and then


should provide this.


if it's not the first minute of of the meeting that that's okay with me.


but in in the first minute uh often there are some some strange things going on which which aren't really well for which which aren't really good.


so what what i'd quite like perhaps is to have some five minutes of of of different meetings.


so


somewhere not in the very beginning five minutes okay.


and then i wanted to ask you just for my information.


then would you be


because i don't quite


so would you be training then um the segmenter so that it could on the basis of that segment the rest of the meeting?


so if i give you like five minutes is the idea that this would then be applied to uh to providing tighter time bands?


i i could do a a retraining with that yeah.


wow interesting.


that's but but i hope that i i don't need to do it.


okay.


uhhuh.


so uh it can be do in an unsupervised way.


excellent.


excellent okay.


i'm i'm not sure.


but for for for those three meetings which i which i did it seems to be quite well.


but there are some some as i said some problems with the lapel mike.


but perhaps we can do something with with cross correlations to to get rid of the of those.


and


yeah.


that's that's what i that's my future work.


well well what i want to do is to to look into cross correlations for for removing those false overlaps.


wonderful.


are the um wireless different than the wired mikes at all?


i mean have you noticed any difference?


i'm i'm not sure um if if there are any wired mikes in those meetings or uh i have have to have a look at them.


but i'm i'm i think there's no difference between.


so it's just the lapel versus everything else?


okay so then if that's five minutes per meeting we've got like twelve minutes twelve meetings roughly that i'm that i've been working with then.


of of of the meetings that you're working with how many of them are different


no.


are there any of them that are different than these two meetings?


well oh in terms of the speakers or the conditions or the


yeah speakers.


sorry.


so


um we have different combinations of speakers.


yeah that


i mean just from what i've seen uh there are some where um you're present or not present.


and then then you have the difference between the networks group and this group.


yeah i know some of the n s a meetings yeah.


yeah.


so i didn't know in the group you had if you had


so you have the networks meeting?


yep we do.


do you have any of jerry's meetings in your pack uh


um no.


no.


we could


i mean you you recorded one last week or so.


i could get that new one in this week i get that new one in.


yep.


we're going to be recording them every monday.


because i think he really needs variety.


so


and and having as much variety for speaker certainly would be a big part of that i think.


great.


okay so if i


okay included include


okay then uh if i were to include all together samples from twelve meetings that would only take an hour.


and i could get the transcribers to do that.


right?


i mean what i mean is that would be an hour sampled and then they'd transcribe those that hour.


that's what i should do?


yeah.


and


right.


but you're


that's that's


i don't mean transcribe.


i mean i mean adjust.


so they get it into the multi channel format and then adjust the timebands so it's precise.


so that should be faster than the ten times kind of thing.


absolutely.


i did i did um uh so last night i did uh


oh gosh.


well last night i did about half an hour in three hours which is not terrific.


but um


anyway it's an hour and a half per


well that's probably


so


well i can't calculate on my on my feet.


do the transcribers actually start with uh transcribing new meetings or are they


well um they're still working they still have enough to finish that i haven't assigned a new meeting.


okay.


but the next


i was about to need to assign a new meeting.


and i was going to take it from one of the new ones.


and i could easily give them jerry feldman's meeting.


no problem.


and then


so they're really running out of data i mean that's good.


uhhuh.


uh that first set.


um okay.


they're running out of data unless we make the decision that we should go over and start uh transcribing the other set.


there the first the first half.


and so i was in the process of like editing them but this is wonderful news.


yeah.


all right.


we funded the experiment with uh


also we were thinking maybe applying that that to getting the


yeah that'll be very useful to getting the overlaps to be more precise all the way through.


so this blends nicely into the update on transcripts.


yes it does.


okay.


so um um liz and and don and i met this morning in the barco room with the lecture hall.


yeah please.


go ahead.


and this afternoon.


it drifted into the afternoon uh concerning this issue of um the well there's basically the issue of the interplay between the transcript format and the processing that they need to do for the s r i recognizer.


and um well so i mentioned the process that i'm going through with the data.


so you know i get the data back from the well uh metaphorically get the data back from the transcriber.


and then i check for simple things like spelling errors and things like that.


and um i'm going to be doing a more thorough editing with respect to consistency of the conventions.


but they're they're generally very good.


and then i run it through uh the channelize program to get it into the multi channel format okay.


and the what we discussed this morning i would summarize as saying that um these units that result in a a particular channel and a particular timeband at at that level um vary in length.


and um their recognizer would prefer that the units not be overly long.


but it's really an empirical question whether the units we get at this point through just that process i described might be sufficient for them.


so as a first pass through a first chance without having to do a lot of hand editing what we're going to do is i'll run it through channelize give them those data after i've done the editing process and be sure it's clean.


and i can do that pretty quickly with just that minimal editing without having to hand break things.


uhhuh.


and then we'll see if the units that we're getting uh with the at that level are sufficient.


and maybe they don't need to be further broken down.


and if they do need to be further broken down then maybe it just be piece wise maybe it won't be the whole thing.


so that's that's what we were discussing this morning as far as i among


right.


also we discussed some adaptational things.


then lots of


so it's like


uh you know i hadn't uh incorporated a convention explicitly to handle acronyms for example.


but if someone says p z m it would be nice to have that be directly interpretable from the transcript what they said.


or uh tcl t c l i mean.


it's like


it's and so um i've i've incorporated also convention with that.


but that's easy to handle at the post editing phase.


and i'll mention it to transcribers for the next phase.


but that's okay.


and then a similar uh convention for numbers.


so if they say one eighty three versus one eight three.


um and also i'll be um encoding as i do my post editing the things that are in curly brackets which are clarificational material.


and uh to incorporate uh keyword at the beginning.


so it's going to be either a gloss or it's going to be a vocal sound like a laugh or a cough or so forth.


or a non vocal sound like a door slam and that can be easily done with a you know just a one little additional thing in the in the general format.


yeah we we just needed a way to strip you know all the comments all the things the that linguist wants but the recognizer can't do anything with.


um but to keep things that we mapped to like reject models or you know uh mouth noise or cough.


and then there's this interesting issue jane brought up.


which i hadn't thought about before.


but i was realizing as i went through the transcripts that there are some noises like


um


well the good example was an in breath where a transcriber working from the mixed signal doesn't know whose breath it is.


right.


and they've been assigning it to someone that may or may not be correct.


and what we do is if it's a breath sound you know a sound from the speaker we map it to a noise model like a mouth noise model in the recognizer.


and yeah it probably doesn't hurt that much once in a while to have these.


but if they're in the wrong channel that's not a good idea.


and then there's also things like door-slams that's really in no one's channel.


they're like it's in the room.


yeah.


and uh jane had this nice uh idea of having like an extra uh couple tiers


an extra channel.


i've been i've been adding that to the ones i've been editing.


and we were thinking that is useful also when there's uncertainties.


so if they hear a breath and they don't know who breath it is it's better to put it in that channel than to put it in the speaker's channel.


because maybe it was someone else's breath.


or uh


so i think that's a good you can always clean that up post processing.


so a lot of little details.


but i think we're coming to some kind of closure on that.


so the idea is then uh don can take uh jane's post processed channelized version and with some scripts you know convert that to to a reference for the recognizer.


and we can can run these.


so when that's ready you know as soon as that's ready and as soon as the recognizer is here we can get twelve hours of force aligned and recognized data.


and you know start working on it.


and


so we're i don't know a a week or two away i would say from.


uh if if that process is automatic once we get your post process transcript.


uhhuh.


and that doesn't the amount of editing that it would require is not very much either.


i'm just hoping that the units that are provided in that way will be sufficient.


because i would save a lot of uh time dividing things.


yeah some of them are quite long.


just from


i don't know.


how long were


you did one


i saw a couple around twenty seconds.


and that was just without looking too hard for it.


so i would imagine that there might be some that are longer.


right.


well


one question.


would that be a single speaker or is that multiple speakers overlapping?


no no but if we're going to segment it


like if there's one speaker in there that says okay or something right in the middle it's going to have a lot of dead time around it.


right it's not the it's not the fact that we can't process a twenty second segment it's the fact that there's twenty seconds in which to place one word in the wrong place.


so it's not


yeah.


you know if if someone has a very short utterance there.


and that's where we might want to have this individual you know have your pre process input.


yep.


sure.


and i just don't know.


i i i thought that perhaps the transcribers could start then from the those multi channel uh speech-nonspeech detections if they would like to.


that's very important.


i have to run it.


right.


in in doing the hand marking?


yeah that's what i was thinking too.


so that's probably what will happen.


but we'll try it this way and see.


i mean it's probably good enough for force alignment.


if it's not then we're really then we definitely


uh but for free recognition i'm it'll probably not be good enough.


we'll probably get lots of errors because of the cross talk and noises and things.


good.


i think that's probably our agenda or starting up there.


oh i wanted to ask one thing.


yeah?


the microphones the new microphones.


when do we get uh


uh they said it would take about a week.


oh exciting.


k.


you ordered them already?


uhhuh.


great.


so what happens to our old microphones?


they go where old microphones go.


um


do we give them to someone?


or


well the only thing we're going to have extra for now


we don't have more receivers.


we just have


right we


right.


so the only thing we'll have extra now is just the lapel.


not not the bodypack.


just the lapel.


just the lapel itself.


um and then one of the one of those.


since what i decided to do on morgan's suggestion was just get two new microphones um and try them out.


uhhuh.


and then if we like them we'll get more.


okay.


yeah.


since they're they're like two hundred bucks a piece.


we won't uh at least try them out.


so it's a replacement for this headset mike.


yep yep.


and they're going to do the wiring for us.


what's the um style of the headset?


it's um it's by crown.


and it's one of these sort of mount around the ear thingies.


and uh when i when i mentioned that we thought it was uncomfortable he said it was a common problem with the sony.


and this is how apparently a lot of people are getting around it.


huh.


and i checked on the web and every site i went to raved about this particular mike.


it's apparently comfortable and stays on the head well.


so we'll see if it's any good.


but uh i think it's promising.


you said it was used by aerobics instructors?


yup.


yep yep so it was it was advertised for performers.


that says a lot.


for the for the record adam is not a paid employee or a consultant of crown.


and


excuse me?


oh.


right.


i said for the record adam is is not a paid consultant or employee of crown.


that's right.


however he may be solicited after these meetings are distributed.


well we're using the crown p z ms.


don't worry about finishing your dissertation.


yeah.


these are crown.


aren't they?


the p z ms are crown.


all right.


yeah i thought they were.


you bet you bet.


and they work very well.


yes.


so if we go to a workshop about all this this it's going to be a meeting about meetings about meetings.


okay.


so


and then it we have to go to the planning session for that workshop.


oh yeah what which will be the meeting about the meeting about the meeting.


because then it would be a meeting about the meeting about the meeting about meetings.


just start saying m four.


yeah okay.


yeah.


m to the fourth.


should we do the digits?


yep go for it.


all right.


so are you


are we going?


it is uh must be february fifteenth.


i think the date's written in there yep.


and actually if everyone could cross out the r nine next to session and write m r eleven.


we didn't have a front end meeting today.


and let's remember also to make sure that one's gets marked as unread unused.


okay.


m r eleven?


m r eleven.


that sounds like a spy code.


huh.


so


there's lots of clicking i'm sure as i'm trying to get this to work correctly.


agenda.


any agenda items today?


i want to talk a little bit about getting how we're going to to get people to edit bleeps parts of the meeting that they don't want to include.


what i've done so far and i want to get some opinions on how to how to finish it up.


i want to ask about um some audio monitoring on some of the um well some of the equipment.


in particular the


well uh that's just what i want to ask.


okay audio monitoring jane.


based on some of the uh


in listening to some of these meetings that have already been recorded there are sometimes big spikes on particular things.


and in pact in fact this one i'm talking on is one of of the ones that showed up in one of the meetings.


so i


oh really?


uhhuh.


yeah.


spikes you mean like uh instantaneous click type spikes or


spikes?


clicks.


huh.


and i don't know what the electronics is but


well i think it's


touching.


uh it it could be a number of things.


it could be touching and fiddling.


and the other thing is that it could


the fact that it's on a wired mike is suspicious.


it might be a connector.


oh okay.


well maybe


then we don't really have to talk about that as an


i i take that off the agenda.


you could try an experiment and say okay i'm about to test for spikes.


and then wiggle the thing there.


and then go and when they go to transcribe it it could ask them to come and get you.


yeah.


right.


oh that


come get me when you transcribe this and see if there's spikes.


um


well okay.


no i'm just


i mean were this a professional audio recording what we would do what you would do is in testing it is you would actually do all this wiggling and make sure that that that things are not giving that kind of performance.


and if they are then they can't be used.


so


um let's see.


i guess i would like to have a discussion about you know where we are on uh recording transcription.


you know basically you know where we are on the corpus.


good.


and then um the other thing which i would like to talk about which is a real meta quest i think deal is uh agendas.


so maybe i'll i'll start with that actually.


uh um andreas brought up the fact that he would kind of like to know if possible what we were going to be talking about.


because he's sort of peripherally involved to this point.


and if there's going to be a topic about discussion about something that he uh strongly cares about then he would come.


and and i think part of part of his motivation with this is that he's trying to help us out.


in the because of uh the fact that the meetings are are tending to become reasonably large now on days when everybody shows up.


and so he figures he could help that out by not showing.


and and i'm sure help out his own time.


huh.


by not showing up if it's a meeting that he's he's


so uh in order


i'd i think that this is a wish on his part.


uh it's actually going to be hard.


because it seems like a lot of times uh things come up that are unanticipated.


and and


but um we could try anyway.


right.


uh do another try at coming up with the agenda uh at some point before the meeting.


uh say the day before.


well maybe it would be a good idea for one of us to like on wednesday or tuesday send out a reminder for people to send in agenda items.


yeah.


okay.


you you want to volunteer to do that?


sure.


all right so we'll send out agenda request.


uh


let me


that'll be


i think that'll help


i'll put that on my spare brain or it will not get done.


that'll help a lot actually.


yeah i have to tell you for the uh for the admin meeting that we have lila does that um every time before an admin meeting.


and uh she ends up getting the agenda requests uh uh ten minutes before the meeting.


but but but


uh but we can try.


maybe it'll work.


huh.


maybe.


yeah.


weirder things have happened.


i'm wondering if he were to just uh specify particular topics.


i mean


maybe we'd be able to meet that request of his a little more.


i would i would also guess that as we get more into processing the data and things like that there'll be more things of interest to him.


well then


actually it


this this maybe brings up another topic which is um


so we're done with that topic.


the other topic i was thinking of was the status on microphones and channels and all that.


yeah actually i i was going to say we need to talk about that too.


why why don't we do that?


okay.


um the new microphones the two new ones are in.


um and they are being assembled as we speak i hope.


and i didn't bring my car today so i'm going to pick them up tomorrow.


um and then the other question i was thinking about is well a couple things.


first of all if the other headsets are a lot more comfortable we should probably just go ahead and get them.


so we'll have to evaluate that when they come in.


and get people's opinions on on what they think of them.


uhhuh.


um then the other question i had is maybe we should get another wireless.


another wireless setup.


i mean it's expensive but it does seem to be better than the wired.


so how many channels do you get to have in a wireless setup?


um well i'm pretty sure that you can daisy chain them together.


so what we would do is replace the wired mikes with wireless.


so we currently have one base station with six wireless mike.


possibility of six wireless receivers.


and apparently you can chain those together.


and so we could replace our wired mikes with wireless if we bought another base station and more wireless mikes.


so um


and


so let's see we


so you know it's still it's fifteen minus six.


right?


so we could have up to nine.


and right now we can have up to six.


right.


and we have five we're getting one more.


yeah.


and it's um about nine hundred dollars for the base station.


and then eight hundred per channel.


oh.


so yeah so the only


beyond the mike the cost of the mikes the only thing is the base station that's nine hundred dollars.


oh we should do it.


okay.


okay so i'll look into how you daisy chain them and and then just go ahead and order them.


i don't quite understand how that how that works.


if


so we're not increasing the number of channels.


no we're just replacing the wired the two wired that are still working.


i see.


along with a couple of the wired that aren't working one of the wired that's not working with a wireless.


basically we found


three wireds work.


right?


i i guess three wireds work yeah.


yeah.


but we've had more problems with that.


yep.


and that sort of bypasses the whole the whole jimbox thing and all that.


right.


and so um we we seem to have uh a reliable way of getting the data in which is through the sony radio mikes.


as long as we're conscious about the batteries.


that seems to be the key issue.


everyone's battery okay?


i checked them this morning.


they should be.


okay.


um because that's the only thing with them.


but the quality seems really good.


and


um i heard from u w that they're they're uh very close to getting their uh setup purchased.


they're they're they're buying something that you can just sort of buy off the shelf.


well we should talk to them about it because i know that s r i is also in the process of looking at stuff.


and so you know what we should try to keep everyone on the same page with that.


yeah.


s r i?


really?


oh.


they got


well maybe this needs to be bleeped out?


i have no clue.


uh i don't know.


i don't know how much of it's public.


probably we shouldn't probably we shouldn't talk about funding stuff.


right.


but anyway there's there's there's uh uh other activities that are going on there and and uh and nist and u w.


so


um but but yeah i i think that at least the message we can tell other people is that our experience is is quite positive with the sony.


uh radio mikes.


now the one thing that you have said that actually concerns me a little is you're talking about changing the headsets.


meaning changing the connector which means some hand soldering or something.


right?


uh no.


no?


we're having the them do it.


so it's hand soldering it but i'm not doing it.


oh.


okay.


so they they charge.


nothing against you and your hand soldering.


but


you've never seen my hand soldering.


but uh as i said they're coming in.


uh okay so that's being done professionally


and


yeah.


i i mean


i mean


as professionally as i guess you can get it done.


well it could


if they do a lot of it it's


i mean it's just their repair shop.


right?


their maintenance people.


well we'll see what it it's like.


yep.


that that can be quite good.


this


yeah okay.


good.


so let's go with that.


uh


and i mean we'll see tomorrow you know what it looks like.


yeah.


so um uh dave isn't here but he was going to start working on some things with the digits.


uh so he'll be interested in what's going on with that.


i guess


was the decision last time was that the the uh transcribers were going to be doing stuff with the digits as well?


uhhuh.


has that started or is that


uh it would be to use his interface and i was going to meet with him today about that.


right so the decision was that jane did not want the transcribers to be doing any of the paperwork.


so i did the all that last week.


so all the all the forms are now on the computer.


and uh then i have a bunch of scripts that we'll read those and let the uh transcribers use different tools.


and i just want to talk to jane about how we transition to using those.


so he has a nice set up that they


it it will be efficient for them to do that.


okay.


so anyway


i i don't think it'll take too long.


so you know just uh a matter of a few days i suspect.


so anyway i think we we have at least one uh user for the digits once they get done which will be dave.


right.


i've already done five or six sets.


so if he wanted to you know just have a few to start with he could.


yeah he might he might be asking


you know and i also have a bunch of scripts that will like generate p files and run recognition on them also.


uh is dave


i don't know if dave is on the list.


if he's invited to these meetings


uh


i don't tend to get an invitation myself for them even.


if he knows


no no.


uh we don't have a active one.


but i'll make sure he's on the list.


yeah.


should we call him?


i mean is he is he definitely not available today?


i don't know.


should i call his office and see?


uh well it's uh


he was in.


i mean he's still taking classes so uh he may well have conflicts.


yeah he was in


he wasn't there at


yeah so this might be a conflict for him.


okay.


uh so


yeah didn't he say his signal processing class was like tuesdays and thursdays?


i think he has a class.


he might have.


oh okay.


you talking about david gelbart?


oh well whatever.


yeah.


yes.


yeah i think he's taking two twenty five a which is now.


so


so that's why we're not seeing him.


uh transcriptions uh beyond the digits where we are and so on.


and the and the recordings also.


um


just where we are.


well so um should we we don't want to do the recording status first or


well we have about thirty two hours.


uh as of i guess a week and a half ago.


so we probably now have about thirty five hours.


and and that's that's uh


how much of that is digits?


it's uh


that's including digits.


right?


i haven't separated it out so i have no clue how much of that is digits.


yeah.


so anyway there's at least probably thirty hours.


or something of


there's got to be more than thirty hour


it couldn't


of of non digits?


of of non digits.


huh.


yeah absolutely.


yeah yeah.


i mean the digits don't take up that much time.


okay.


okay and the transcribers


i uh don't have the exact numbers.


but i think it would come to about eleven hours that are finished uh transcribing from them right now.


the next step is to that i'm working on is to insure that the data are clean first and then channelized.


what i mean by clean is that they're spell checked.


that the mark up is consistent all the way throughout.


and also that we now incorporate these additional conventions that uh liz requested in terms of um um in terms of having a a systematic handling of numbers and acronyms which i hadn't been specific about.


um for example they'll say uh ninety two.


and you know so how you could


nine two.


exactly.


right.


so if you just say nine two the there are many ways that could have been expressed.


and i just had them


i i mean a certain number of them did put the words down.


but now we have a convention which also involves having it followed by um a gloss and things.


you know jane?


uhhuh.


um one suggestion


and you may already be doing this.


but i've noticed in the past that when i've gone through transcriptions and you know in in order to build lexicons and things


if you um just take all the transcriptions and separate them into words and then alphabetize them a lot of times just scanning down that list you'll find a lot of inconsistencies and


misspelled.


yeah.


you're talking about the type token frequency listings and i use those too.


you mean just uh on each on each line there's a one word.


right?


it's one token from the from the corpus.


yeah those are extremely efficient.


and and i i agree that's a very good use of it.


oh so you already have that.


okay.


well that's that's a way that's


you know the spell check basically does that.


but but in addition


yes that's that's exactly the strategy i want to do in terms of locating these things which are you know colloquial spoken forms which aren't in the lexicon.


uhhuh.


exactly.


because a lot of times they'll appear next to each other and uh


and then you then you can do a


yeah.


in alphabetized lists they'll appear next to each other and and so it makes it easier.


absolutely.


i agree.


that's a very good that's a very good uh suggestion.


and that was that's my strategy for handling a lot of these things in terms of things that need to be glossed.


i didn't get to that point.


but


so there are numbers then there are acronyms.


and then um there's a she wants the uh actually a an explicit marker of what type of comment this is.


so curly inside the curly brackets i'm going to put either voc for vocalized.


like cough or like laugh or whatever.


nonvoc for door slam.


and gloss for things that have to do with if they said a a spoken form with this this pronunciation error.


right.


i already had that convention.


but i i haven't been asking these people to do it systematically.


oh that's great.


because i think it most most efficiently handled by uh by a a filter.


that was what i was always planing on.


so that you know you get a whole long list


exactly what you're saying.


you get a whole list of things that say curly bracket laugh curly bracket.


uhhuh.


then you know it's it's


you you risk less error if you handle it by a filter than if you have this transcriber laboriously typing in sort of a voc space


yeah.


so so many ways that error prone.


so um um i'm i'm going to convert that via a filter into these tagged uh subcategorized comments.


and same thing with you know we see you get a subset when you do what you're saying.


you end up with a with uh you're collapsing across a frequency you just have the tokens.


and you can um have a filter which more efficiently makes those changes.


but the numbers and acronyms have to be handled by hand.


because you know i mean


you don't know what they could be.


yeah now timit's clear um and p l p is clear.


but uh there are things that are not so well known in


or or have variant uses like the numbers you can say nine two or you can say ninety two.


and uh


so how are you doing the


i'd handle the numbers individually.


how are you doing the uh acronyms so if i say p z m what would it appear on the transcript?


it would be separate.


the letters would be separated in space.


okay.


and potentially they'll have a curly bracket thing afterwards.


but i'm not sure if that's necessary clarifying what it is.


uhhuh.


so gloss of whatever.


right.


i don't know if that's really necessary to do that.


maybe it's a nice thing to do because of it then indicating this is uh a step away from indicating that it really is intentional that those spaces are there.


and indicating why they're there to indicate that it's uh the you know uh enumerated or


it's not a good way of saying.


but it's it's the specific uh way of stating these these letters.


so it sounds good.


and so anyway the clean those are those things.


and then channelized is to then um get it into this multichannel format.


and at that point then it's ready for use by liz and don.


but that's been my top priority beyond getting it channelized.


the next step is to work on tightening up the boundaries of the time bins.


yeah.


and uh thilo had a a breakthrough with this this last week in terms of getting the channel based um uh speech-nonspeech segmentation um up and running.


and i haven't i haven't been able to use that yet.


because i'm working


this is my top priority get the data clean and channelized.


i actually gave


have you also been doing spot checks jane?


oh yes.


okay good.


well you see that's part of the cleaning process.


i spent um


actually um i have a segment of ten minutes that was transcribed by two of our transcribers.


oh good.


good.


and i went through it last night it's it's almost spooky how similar these are word for word.


and there are some differences in commas because commas i i left them discretion at commas.


right.


uh and so


because it's not part of our of our needed conventions.


uhhuh.


and um


and so they'll be a difference in commas.


but it's word by word the same in in huge patches of the data.


and i have ten minute stretch where i can where i can show that.


and and sometimes it turns out that one of these transcribers has a better ear for technical jargon.


and the other one has a better ear for colloquial speech.


so um the one the colloquial speech person picked up gobbledy gook.


huh.


and the other one didn't.


and on this side this one's picking up things like neural nets and the one that's good on the on the vocabulary on the uh colloquial didn't.


when


for the person who missed gobbledy gook what did they put?


it was an interesting approximation put in parentheses.


because i have this convention that if they're not sure what it was they put it in parentheses.


oh.


so they tried to approximate it but it was


oh good.


it was spelled g a b b l


sort of how it sounds?


yes.


more of an attempt to


yeah.


i mean apparently it was very clear to her that these this this was a sound these are the sounds.


it was a technical term that she didn't recognize.


but


but she knew that she didn't know it.


maybe it was a technical


exactly.


but she even though her technical perception is just really uh you


know i've i'm tempted to ask her if she's taken any courses in this area or if she's taken cognitive science courses.


right.


then because neural nets and oh she has some things that are oh down sampled she got that right.


huh.


and some of these are rather uh unexpected.


obscure.


but ten solid uh chunk of ten solid minutes where they both coded the same data.


and um


and and again the main track that you're working with is eleven hours?


is that right?


yes exactly.


yeah okay.


and that's part of this


eleven hours.


is that is that that including digits?


yes it is.


yeah.


so let's say roughly ten hours or so of


uhhuh.


i mean it's probably more than that but but with of of non digits.


it'd be more than that.


because i my recollection is the minutes that digits don't take more than half a minute per person.


oh okay.


but um the the total set that i gave them is twelve hours of tape.


oh i see.


but they haven't gotten to the end of that yet.


so they're still working some of them are two of them are still working on completing that.


boy they're moving right along.


yeah they are.


uhhuh they're very efficient.


there're some who have more hours that they devote to it than others.


so what what what's the deal with with your


the channel thing?


yeah.


oh it's just uh i ran the recognizer uh the speech-nonspeech detector on different channels.


and it's just in uh in this new multi channel format and output.


and i just gave one one meeting to to liz who wanted to to try it for for the recognizer.


as uh apparently the recognizer had problems with those long chunks of speech which took too much memory or whatever.


right.


and so she she will try that i think.


and


i'm i'm working on it.


so i hope


is this anything different than the h m m system you were using before?


no.


uh i huh use some some different features but not not


uhhuh.


the basic thing is this h m m base.


so there's still no no knowledge using different channels at the same time.


you know what i mean?


there is some uh as the energy is normalized across channels.


across all of them.


yeah.


okay.


so


but basically that's one of the main changes.


what are some of the other features besides the energy?


you said you're trying some different features or something.


oh i just uh huh i just use um our loudness based things.


now as they


before there were they were some in in the log domain and i i changed this to the to the


cube root.


to


no i changed this to the to the to the loudness thingy with the with the


huh.


uh.


how do you call it?


i'm not sure.


with the uh


fletcher munson?


no.


i'm not sure about the term.


oh okay.


uh i'll look it up.


yeah all right.


and say it to you.


uh okay and


yeah.


that's that's basically the the the thing.


okay.


yeah and i and i tried to normalize uh uh the features there's loudness and modified loudness um within one channel.


because they're yeah to to be able to distinguish between foreground and background speech.


and it works quite well.


but not always.


uhhuh.


so


good.


um let's see.


i think the uh


were were you basically done with uh transcriptions or


so i guess the next thing is this uh bleep editing.


right.


so the the idea is that we need to have


we need to provide the transcripts to every participant of every meeting to give them an opportunity to bleep out sections they don't want.


so i've written a bunch of tools that will generate web pages uh with the transcription in it so that they can click on them and piece pieces and they can scroll through and read them.


and then they can check on each one if they want it excluded.


and then it's a form h t m l form so they can submit it and it will end up sending me e mail with the times that they want excluded.


and so uh some of the questions on this is what do we do about the privacy issue.


yeah.


and so i thought about this a little bit and i think the best way to do it is every participant will have a password.


a single password.


each person will have a single password user name and password.


and then each meeting we'll only allow the participants who were at that meeting to look at it.


and that way each person only has to remember one password.


i i can't help but wonder if this is maybe a little more elaborate than is needed.


i mean if people have


uh


i mean for me i would actually want to have some pieces of paper that had the transcription and i would sort of flip through it.


and then um if i thought it was okay i'd say it's okay.


uhhuh.


and i


uh i mean it depends how this really ends up working out.


but i guess my thought was that the occasion of somebody wondering whether something was okay or not and needing to listen to it was going to be extremely rare.


right i mean so the fact that you could listen to it over the web is a minor thing that i had already done for other reasons.


okay.


and so that that's a minor part of it.


i just wanted some web interface so that people you didn't actually have to send everyone the text.


so what my intention to do is that as the transcripts become ready um i would take them and generate the web pages.


and send email to every participant or contact them using the contact method they wanted.


and just uh tell them here's the web page.


um you need a password.


so question number one is how do we distribute the passwords?


and question number two is how else do we want to provide this information if they want it?


that's


i think what i was sort of saying is that if you just say here is a here is


i mean this maybe it sounds paleolithic but but i just thought if you handed them some sheets of paper that said uh here's what was said in this transcription is it okay with you.


and if it is here's this other sheet of paper that you sign that says that it's okay.


i think that um there are a subset of people who will want printouts that we can certainly provide.


and then they'd hand it back to you.


but certainly i wouldn't want a printout.


these are big.


and i would much rather be be able to just sit and leaf through it.


you find it easier to go through a large


i mean how do you read books?


well i certainly read books by hand.


but for something like this i think it's easier to do it on the web.


really?


i mean it


because you're going to get you know if i i'm i'm in a bunch of meetings and i don't want to get a stack of these.


i want to just be able to go to go to the web site and visit it as i want.


going to a web site is easy.


but flipping through a hundred pounds a hundred pages of stuff is not easy on the web.


well i don't think it's that much harder than paper.


i have one question.


so


so are you thinking that um the person would have a transcript and go strictly from the transcript?


because i i do think that there's a benefit to being able to hear the tone of voice and the


so here's the way i was imagining it and maybe i'm wrong.


yeah.


but the way i imagined it was that um the largest set of people is going to go oh yeah i didn't say anything funny in that meeting.


just go ahead.


where's the where's the release?


and then there'll be a subset of people.


right?


okay there's


i mean think of who it is we've been recording mostly.


okay there'll be a subset of people who um will say uh well yeah i really would like to see that.


and for them the easiest way to flip through if it's a really large document


i mean unless you're searching.


searching of course should be electronic.


but if you're not


so if you provide some search mechanism you go to every place they said something or something like that.


but see then we're getting more elaborate with this thing.


um if if uh you don't have search mechanisms you just sort of have this really really long document.


i mean whenever i've had a really really long document that it was sitting on the web i've always ended up printing it out.


i mean so it's it's


i mean you you're you're not necessarily going to be sitting at the desk all the time.


you want to figure you have a train ride and there's all these situations where where i


i mean this is how i was imagining it anyway.


and then i figured that out of that group there would be a subset who would go huh you know i'm really not sure about this section here.


and then that group would need it


it seems like


if i'm right in that it seems like you're setting it up for the most infrequent case rather than for the most frequent case.


so that uh now we have to worry about privacy.


well no for the most


we have to worry about all these passwords for different people.


for the most frequent case they just say it's okay.


and then they're done.


and i think almost everyone would rather do that by email than any other method.


uhhuh.


the other thing too is it seems like


um yeah that's true.


go ahead.


i mean because you don't have to visit the web page if you don't want to.


yeah.


i guess


yeah i guess we don't need their signature.


i guess an email okay is all right.


oh that was another thing.


i i had assumed that we didn't need their signature that it that an email approval was sufficient.


but i don't actually know.


are are people going to be allowed to bleep out sections of a meeting where they weren't speaking?


yes.


i also


if someone feels strongly enough about it then i i i think they should be allowed to do that.


so that means other people are editing what you say?


uh i don't know about that.


i don't know if i like that.


well the only other choice is that the person would say no don't distribute this meeting at all.


and i would rather they were able to edit out other people then just say don't distribute it at all.


but what they signed in the consent form was something that said you can use my voice.


well but if if someone is having a conversation and you only bleep out one side of it that's not sufficient.


right?


yeah.


yeah but that's our decision then.


um i don't think so.


i think it is.


i mean because if i object to the conversation


if i say we were having a conversation and i consider that conversation private and i consider that your side of it is enough for other people to infer i want to be able to bleep out your side.


the


i agree that the consent forms were uh i


i agree with what adam's saying that um the consent form did leave open this possibility that they could edit things which they found offensive whether they said them or didn't say them.


i see.


okay well if that's what it said.


and the other thing is from the standpoint of the of the


i'm not a lawyer.


but it strikes me that uh we wouldn't want someone to say oh yes i was a little concerned about it.


but it was too hard to access.


so i think it's kind of nice to have this facility to listen to it.


now in terms of like editing it by hand i mean i think it's some people would find that easier to specify the bleep part by having a document they edited.


but but it seems to me that sometimes um you know if a person had a bad day and they had a tone in their voice that they didn't really like you know it's nice it's nice to be able to listen to it and be sure that that was okay.


i mean i can certainly provide a printable version if people want it.


um i mean it's also a mixture of people.


um


i mean some people are do their work primarily by sitting at the computer flipping around the web.


and others do not.


yep.


others would consider it


this uh a a set of skills that they would have to gain.


you know?


well i think most of the people in the meetings are the former.


it depends on what meetings.


that's true.


so far.


so


in the meetings so far yeah.


but we're trying to expand this.


right?


right.


so i i i actually think that paper is the more universal thing.


and that


uhhuh.


well but if they want to print it out that's all right.


yeah.


i think everyone in the meeting can access the web.


no i think we have to be able to print it out.


it's not just if they want to print it out.


okay so does that mean that i can't use email?


i i think


or what?


because you could send it through email you're thinking.


i i


well we


well i don't think i


there was this


well i don't think we can send the text through email because of the privacy issues.


good.


no.


for security?


yeah okay good.


yeah.


right.


good point.


um so giving them you think a web site to say if you want to print it out here it is is not sufficient?


i


certainly for everybody who's been in the meetings so far it would be sufficient.


i'm just wondering about


yeah i'm just thinking for people that that's not sufficient for what the only sufficient thing would be for me to walk up to them and hand it to them.


you could mail it to them.


get a mailing address.


equivalent.


but i think it's easier to drop in the box.


but


just put the button on on the web page which say please send me the the scripts.


oh that's interesting.


that's right.


what um


when you display it on the web page what are what are you showing them?


utterances?


uhhuh.


or


and so can they bleep within an utterance?


no.


whole utterances only.


whole utterances.


and that was just convenience for my sake that it's uh uh


it would end up being fairly difficult to edit the transcripts if we would do it at the sub utterance level.


because this way i can just delete an entire line out of a transcript file rather than have to do it by hand.


there's another aspect to this which maybe is part of why this is bothering me.


um i think you're really trying very hard to make this as convenient as possible for people to do this.


i mean that's why i did the web form because for me that would be my most convenient.


huh.


i i i understand.


i think that's the bad idea.


i know where you're going.


oh.


see because you're you're uh


really!


you're going to end up with all these little patchy things whereas really what we want to do is have the the the bias towards letting it go.


because you


know it


there was a


one or once or twice in the in the meetings we've heard where somebody said something that they might be embarrassed by.


but overall people are talking about technical topics.


nobody's going to get hurt.


nobody's being libeled.


you know this is this we're we're covering


we're playing the lawyer's game and we're playing we're we're we're looking for the extreme case.


if we really orient it towards that extreme case make it really easy we're going to end up encouraging a headache.


that i think that's


i'm sort of psyching myself out here.


i i'm trying to uh


but i i think that's


i guess i don't see having a few phrases here and there in a meeting being that much of a headache bleeped out.


well it's


so


but


i think what morgan's saying is the easier it is the more is going to be bleeped.


and and it really depends on what kind of research you're doing.


i think some researchers who are going to be working with this corpus years from now are really going to be cursing the fact that there's a bunch of stuff in there that's missing from the dialogue.


uhhuh.


you know it depends on the kind of research they're doing.


yeah.


but it might be uh it might be really a a pain.


and you know where it's really going to hurt somebody in some way.


the one who said it or someone who is being spoken about we definitely want to allow the option of it being bleeped out.


but i really think we want to make it the rare incidence.


and and uh i am just a little worried about making it so easy for people to do.


and so much fun that they're going to go through and bleep out stuff.


so much fun.


and they can bleep out stuff they don't like too.


right?


from somebody else as you say.


you know so well i didn't like what he said.


well i don't see any way of avoiding that.


i mean we have to we have promised that we would provide them the transcript and that they can remove parts that they don't like.


so that the


no no i i i don't


the only question is


you've talked me into that.


but i i just think that we should make it harder to do.


the problem is if it's harder for them it's also harder for me.


whereas this web interface i just get email.


it's all formatted.


it's all ready to go and i can just insert it.


so maybe you don't give them access to the web interface unless they really need it.


well i guess


so so so


yeah.


i'm sorry.


so so so maybe this is a a way out of it.


huh.


you've provided something that's useful for you to do handle.


and useful for someone else if they need it.


but i think the issue of privacy and ease and so forth should be that uh they get access to this if they really need it.


well


so you're saying the the sequence would be more like first adam goes to the contact lists contacts them via whatever their preferred method is to see if they want to review the meeting.


right.


and then if they don't you're done.


if they do then he provides them access to the the web site.


well to some extent i have to do that anyway because as i said we have to distribute passwords.


or a printed out form.


there's there


so


but you don't necessarily have to distribute passwords is what i'm saying.


well but


only if they want it.


what i'm saying is that i can't just email them the password because that's not secure.


no no no.


but you aren't necessarily giving them


so they have to call me and ask.


but we don't even necessarily need to end up distributing passwords at all.


huh


well we do because of privacy.


we can't just make it openly available on the web.


no no.


you're missing the point.


uhhuh.


we're we're trying we're trying to make it less of an obvious just uh fall off a log to do this.


not everyone gets a password unless they ask for it.


right?


so so what i would see is that first you contact them and ask them if they would like to review it for to check for the


yeah.


not just for fun.


okay?


but to to check this for uh things that they're worried about having said or if they're willing to just send an approval of it at from their memory.


um and uh and we should think carefully actually we should review go through how that's worded.


then if someone uh wants to review it uh


and i know you don't like this but i'm offering this as a suggestion is that is that we then give them a print out.


and then if they say that i have a potential problem with these things then you you say okay well you might want to hear this in context to think if you need that you issue them a password in the


but the the problem with what you're suggesting is it's not just inconvenient for them it's inconvenient for me.


because that means multiple contacts every time for every single meeting every time anyone wants anything.


i would much prefer to have all be automatic they visit the web site if they want to.


obviously they don't have to.


i know you'd prefer it but the


we have


there's a problem with it.


so i think you're thinking people are going to arbitrarily start bleeping and i just don't think that's going to happen.


i'm also concerned about the spirit of the of the informed consent thing.


because i think if they feel that uh it's i i


you know if it turns out that something gets published in this corpus that someone really should have eliminated and didn't detect then it could have been because of their own negligence that they didn't pursue that next level and get the password and do that.


um but but they might be able to argue oh well it was cumbersome and i was busy and it was going to take me too much time to trace it down.


so it could that the burden would come back onto us.


so i'm a little bit worried about uh making it harder for them from the legal standpoint.


well you can go too far in that direction and you need to find somewhere between i think.


yeah.


because


it seems to me that sending them email saying if you have an o okay reply to this email and say okay.


uhhuh.


if you have a problem with it contact me and i'll give you a password.


seems like is a perfectly reasonable compromise.


and if they want a printout they can print it out themselves.


or we could print it up for them.


i mean we could offer that.


but but there's uh another aspect to that and that is that in the informed consent form um my impression is that they that we offered them at the very least that they definitely would have access to the transcript.


and and i


i don't know that there's a chance of really skipping that stage.


i mean i i thought that you were


maybe i misinterpreted what you said but it's


having access to it doesn't necessarily mean that having it.


having it.


giving it to them.


right?


okay.


it just means they have the right to have it.


well the in


all right.


the consent form is right in there if anyone wants to look at it.


fine.


fair enough.


so


yeah.


you want me to grab one?


well i could


i'm closer.


i could


yeah but you're wired.


aren't you?


that is true.


um yeah i mean i don't want to fool them.


i don't know


i just meant that every any time you say anything to anyone there is in fact a a bias that is presented.


oh yeah yeah.


oh i know.


right?


yeah that's true.


of and


if you agree to participate you'll have the opportunity to have anything anything excised which you would prefer not to have included in the data set.


yeah.


once a transcript is available we will ask your permission to include the data in the corpus for the larger research community.


there again you will be allowed to indicate any sections that you'd prefer to have excised from the data base.


and they will be removed both from the transcript and the recording.


huh.


well that's more open than i realized.


well i mean it


the one question is definitely clear with anything as opposed to just what you said.


i


yeah uh no that it


that's true.


that's more severe but the next one says the transcript will be around.


that's right.


and it doesn't really say we'll send it to you.


or it'll be available for you on the web or anything.


i think it probably leaves it open how we get it to them.


at least it more often.


yeah.


it means also we don't have to to give it to them.


i mean like like morgan was saying they they


they just have to make sure that it is available to them.


it's available to them if they ask for it.


yeah okay.


so um i think i have an idea that may be may satisfy both you and me in this which is um


it's a it we just go over carefully how these notes to people are worded.


so i i just want it to be worded in such a way where it gives the strong it gives very i mean nothing hidden very strongly the bias that we would really like to use all of these data.


right.


that that we really would rather it wasn't a patchwork of things tossed out.


good.


that it would be better for um our uh field if that is the case.


but if you really think something is going to


and i don't think there's anything in the legal aspects that that is hurt by our expressing that bias.


great.


great great.


and then then my concern about which


yeah.


i agree.


you know you might be right it may be it was just paranoia on my part uh but people just


see i'm worried about this interface so much fun that people start bleeping stuff out just as just because they can.


it's just a check box next to the text it's not any fun at all.


well i don't know.


i kind of had fun when you played me something that was bleeped out.


you know?


i


well but they won't get that feedback.


all


oh they won't?


no because it doesn't automatically bleep it at the time.


oh good.


it just sends me


so you haven't made it so much fun.


right.


okay.


it just sends me the time intervals.


and then at some point i'll incorporate them all and put bleeps.


i mean i don't want to have do that yet until we actually release the data.


yeah.


because um then we have to have two copies of every meeting.


and we're already short on disk space.


so i i want to i just keep the times until we actually want to release the data and.


then we bleep it.


all right so i think


yeah so if we have if


again let's you know sort of circulate the the wording on each of these things and get it right.


but but


well since you seem to feel uh strongest about it would you like to do the first pass?


uh fair enough.


turn about is fair play.


also there is this other question the legal question that that adam's raised uh about whether we need a concrete signature or email suffices or whatever.


sorry.


and i don't know how that works.


there's something down there about if you agree to


i'm i'm i'm i thought i i thought about it with one of my background processes.


i don't think so.


and i uh it's uh it's uh it's fine to do the email.


uh.


fine.


good.


okay.


yeah because they're signing here that they're agreeing to the paragraph which says you'll be given an opportunity.


yeah.


and


and so i don't think they need another signature.


well and furthermore i


it's now fairly routine in a lot of arrangements that i do with people on contracts and so forth.


that that uh if it's if it's that sort of thing where you're you're saying uh okay i agree we want eighty hours of this person at such and such amount and i agree that's okay.


uh if it's a follow up to some other agreement where there was a signature it's often done in email now.


right.


so it's it's okay.


great.


um


so i guess i probably should at the minimum think about how to present it in a printed form.


i'm not really sure what's best with that.


the problem is a lot of them are really short.


well


and so i don't necessarily want to do one per line.


well i


but i don't know how else to do it.


i also have this


i i think it's nice you have it uh hearable on the on the web for those who might wonder about um the non nonverbal side.


i mean i i agree that our bias should be as as expressed here.


and but i i think it's nice that a person could check.


because sometimes you know you the words on a on the page come out sounding different in terms of the social dynamics if they hear it.


huh.


uhhuh.


and i realize we shouldn't emphasize that people you know shouldn't borrow trouble.


what it comes down to but


yeah i think actually


my opinion probably is that the only time someone will need to listen to it is if the transcript is uh not good.


you know if if there are lots of mumbles and parentheses and things like that.


oh you know or what if there was an error in the transcript that didn't get detected and there was a whole uh segment against some personal


right.


that was all mumbled?


i think microsoft is


yeah.


yeah exactly.


oh.


sorry transcribers.


or or even or even there was a a line you know about how huh uhhuh bill gates duh duh duh duh


but but it was all the words were all visible but they didn't end up there was a slip in the transcript.


oh god.


they're going to hate this meeting.


yeah that's true.


actually liz will like it.


know but


liz will like it.


we had a pretty strong disagreement going there.


yep yep that's right.


so i don't know.


i mean i i guess we're assuming that the transcript is a close enough approximation and that that my double checking will be so close to absolutely perfect that it that nothing will slip by.


uhhuh.


but it the


something might sometime and they uh


if if it's something that they said they might


i mean you might be very accurate in putting down what they actually said.


but when they hear it themselves they may hear something different because they know what they meant.


i don't know how to notate that.


sarcasm.


yeah that's right.


how do you how do you indicate sarcasm?


no i'm serious.


so the so the


so we might we might get some feedback from people that such and such was you know not not really what i said.


yeah.


well that would be good to get definitely.


yeah but


yeah sure.


just for corrections.


so um in terms of password distribution i think phone is really the only way to do it.


phone and in person.


or mail physical mail.


or if


leave it on their voice mail.


any sub word level thing.


any sub


yeah okay.


i mean you could do it with p g p or things like that but it's too complex.


you know i just realized something which is of this question about the uh the possible mismatch of


i mean


well and actually also the lawyer saying that um we shouldn't really have them have the people believing that they will be cleared by our checks.


you know?


i mean so it's like in a way it's it's nice to have the responsibility still on them to listen to the tape and and hear the transcript.


uhhuh.


to have that be the


well yeah but you can't


i mean most people will not want to take the time to do that though.


yeah okay fair enough.


and they're they're absorbing the responsibility themselves.


and they they have to


so it's not it's not um


yeah good.


but i mean if you were at a meeting and and you you don't think at least that you said anything funny.


and the meeting was about you know some some funny thing about semantics or something or uh


you probably won't listen to it.


yeah.


it is true that that the content is technical.


i and so and we're not having these discussions which


i i mean when i listen to these things i don't find things that are questionable in other people's speech or in my own.


you would think it would be rare.


just


it should be very rare.


i mean we're not talking about the energy crisis or something people have


yeah okay.


how about them energy crises?


i think we're uh


done?


kind of done.


actually i was going to


did you have anything that's going on or


not really.


no.


um my project is going along.


but um i'm really just here to um fill the project uh the overall progress.


yeah.


i don't really have anything specific to to talk about.


that's fine.


i just didn't want to go by you if you had something.


oh okay.


you don't have anything to say.


nah.


oh.


transcribers he was rattling the marbles in his brain back and forth just then this this


shall we do digits?


oh yeah.


um oh by the way i did find a bunch


it um


uh we should count out


how many more digits to forms do we have back there?


there were quite a few.


that's what i thought.


uh


i i was going through them all and i found actually a lot filed in with them that were blanks that no one had actually read.


huh.


and so we still have more than i thought we did.


oh good.


so we have a few more digits before we're done.


okay we're on.


okay.


so i mean everyone who's on the wireless check that they're on?


we


all right.


i see.


yeah.


okay our agenda was quite short.


oh could you close the door maybe?


sure.


two items which was uh digits and possibly stuff on on uh forced alignment.


which jane said that liz and andreas had information on.


uhhuh.


but they didn't.


so


i guess the only other thing uh for which i


we should do that second.


because liz might join us in time for that.


okay.


um


okay so there's digits alignments.


and um


i guess the other thing which i came unprepared for uh is uh to see if there's anything anybody wants to discuss about the saturday meeting.


right.


any i mean maybe not.


digits and alignments.


but


uh


talk about aligning people's schedules.


yeah.


uhhuh.


i mean


yeah i mean it was


yeah it's forced alignment of people's schedules.


if we're very


forced align.


with with whatever it was a month and a half or something ahead of time the only time we could find in common roughly in common was on a saturday.


yep.


ugh.


it's pretty sad.


have have we thought about having a conference call to include him in more of in more of the meeting?


i i mean i don't know.


if we had the if we had the telephone on the table


no.


but i mean he probably has to go do something.


no actually i i have to i have to shuttle kids from various places to various other places.


right?


i see okay.


so


yeah.


and i don't have and i don't um have a cell phone.


a cell phone?


so i can't be having a conference call while driving.


no it's not good.


right.


that's not good.


so we have to we


plus it would make for interesting noise background noise.


yep.


uh


so we have to equip him with a with a with a head mounted uh cell phone.


and we'd have to force you to read lots and lots of digits.


and


oh yeah.


so it could get real real car noise.


take advantage.


and with the kids in the background.


i'll let i'd let


yeah.


i let uh my five year old have a try at the digits.


so anyway i can talk about digits.


um did everyone get the results or shall i go over them again?


i mean that it was basically the only thing that was even slightly surprising was that the lapel did so well.


um and in retrospect that's not as surprising as maybe


it shouldn't have been as surprising as i as as i felt it was.


the lapel mike is a very high quality microphone.


and as morgan pointed out that there are actually some advantages to it in terms of breath noises and clothes rustling if no one else is talking.


exactly.


uhhuh.


um so uh


it's it


well it's yeah sort of the the breath noises and the mouth clicks and so forth like that the lapel's going to be better on.


or the cross talk.


the lapel is typically worse on the on clothes rustling.


but if no one's rustling their clothes


right i mean a lot of people are just sort of leaning over and reading the digits.


it's it's


so it's it's a very different task than sort of the natural.


yeah.


you don't move much during reading digits i think.


so


right.


probably the fact that it picks up other people's speakers other people's talking is an indication of that it the fact it is a good microphone.


right so in the digits in most most cases there weren't other people talking.


do the lapel mikes have any directionality to them?


there typically don't no.


because i i suppose you could make some that have sort of that you have to orient towards your mouth.


and then it would


they have a little bit.


but they're not noise cancelling.


so uh


they're they're intended to be omni-directional.


uhhuh.


and it's and because you don't know how people are going to put them on you know.


so also andreas on that one the the back part of it should be right against your head.


and that will keep it from flopping up and down as much.


it is against my head.


okay.


yeah.


um yeah we actually talked about this in the uh front end meeting this morning too.


much the same thing.


and and it was


uh i mean there the point of interest to the group was primarily that um the uh the system that we had that was based on h.t.k. that's used by you know all the participants in aurora was so much worse than the than the s.r.i.


everybody.


and the interesting thing is that even though yes it's a digits task and that's a relatively small number of words and there's a bunch of digits that you train on it's just not as good as having a a very large amount of data and training up a a a nice good big h.m.m.


um also you had the adaptation in the s.r.i. system which we didn't have in this.


um so um


and we know


did i send you some results without adaptation?


no.


or if you did i didn't include them.


i think stephane uh had seen them.


because it was


so


yeah i think i did actually.


so there was a significant loss from not doing the adaptation.


yeah.


um


a a a couple percent or


i mean


well i don't know it overall


uh i i don't remember.


but there was there was a significant um loss or win from adaptation with with adaptation.


and um


that was the phone loop adaptation.


and then there was a very small like point one percent on the natives uh win from doing um you know adaptation to the recognition hypotheses.


and i tried both means adaptation and means and variances.


and the variances added another or subtracted another point one percent.


so it's um that's the number there.


point six i believe is what you get with both uh means and variance adaptation.


right.


but i think one thing is that uh i would presume


have you ever have you ever tried this exact same recognizer out on the actual t.i. digits test set?


this exact same recognizer?


no.


it might be interesting to do that.


because my my because my sense


um


but but i have i mean people people at s.r.i. are actually working on digits.


i bet it would do even slightly better.


i could and they are using a system that's um you know is actually trained on digits.


um but otherwise uses the same you know decoder the same uh training methods and so forth.


uhhuh.


and i could ask them what they get on t.i. digits.


yeah although i'd be i think it'd be interesting to just take this exact actual system.


so that these numbers were comparable.


well adam knows how to run it.


and try it out on t.i. digits.


yeah no problem.


so you just make a


yeah.


because our sense from the other from the aurora uh task is that


and try it with t.i. digits.


i mean because we were getting sub one percent numbers on t.i. digits also with the tandem thing.


huh.


so one so there were a number of things we noted from this.


one is yeah the s.r.i. system is a lot better than the h.t.k.


this you know very limited training h.t.k. system.


uh but the other is that um the digits recorded here in this room with these close mikes uh are actually a lot harder than the studio recording t.i. digits.


uhhuh.


i think you know one reason for that uh might be that there's still even though it's close talking there still is some noise and some room acoustics.


and another might be that uh i'd i would presume that in the studio uh uh situation recording read speech that if somebody.


did something a little funny or pronounced something a little funny or made a little that they didn't include it.


they didn't include it?


they made them do it again.


whereas i took out the ones that i noticed that were blatant that were correctable.


so that if someone just read the wrong digit i corrected it.


and then there was another one where jose couldn't tell whether i couldn't tell whether he was saying zero or six.


yeah.


and i asked him and he couldn't tell either.


so i just cut it out.


you know so i just edited out the first uh word of the utterance.


um so there's a little bit of correction but it's definitely not as clean as t.i. digits.


so my expectations is t.i. digits would especially


i think t.i. digits is all american english.


right?


so it would probably do even a little better still.


uhhuh.


on the s.r.i. system.


but we could give it a try.


well


but remember we're using a telephone bandwidth front end here uh on this uh on this s.r.i. system.


so um i was i thought that maybe that's actually a good thing.


because it it gets rid of some of the uh the noises.


um you know in the the below and above the um the you know speech bandwidth.


and um


i suspect that to get sort of the last bit out of these higher quality recordings you would have to in fact uh use models that uh were trained on wider band data.


and of course we can't do that or


what's t.i. digits?


i thought


it's wide band yeah.


it's in in fact we looked it up.


it is wide band.


okay.


and it was actually twenty kilohertz sampling.


oh that's right.


i i did look that up.


uhhuh.


i couldn't remember whether that was t.i. digits or one of the other digit tasks.


yeah.


right.


but but i would


it's it's easy enough to try.


just run it on


so morgan you're getting a little breath noise.


see


now uh does


one one issue


you might want to move the mike down a little bit.


one issue with with that is that um the system has this uh notion of a speaker to which is used in adaptation variance uh you know both in uh mean and variance normalization.


and also in the v.t.l. estimation.


so


yeah i noticed the script that extracted it.


do


is


so does so so does does um the t.i. digits database have speakers that are known?


yep.


and is there is there enough data or a comparable comparable amount of data to to what we have in our recordings here?


that i don't know.


i don't know.


i don't know how many speakers there are.


and and how many speakers per utterance.


okay.


yeah.


well the other thing would be to do it without the adaptation and compare to these numbers without the adaptation.


that would


right uh but i'm not so much worried about the adaptation actually than than the um um the uh v.t.l. estimation.


right.


if you have only one utterance per speaker you might actually screw up on estimating the the warping uh factor.


so um


i strongly suspect that they have more speakers than we do.


right but it's not the amount of speakers.


so uh


it's the it's the amount of data per speaker.


right so we we could probably do an extraction that was roughly equivalent.


um


so


so although i i sort of know how to run it there are a little a few details here and there that i'll have to dig out.


the key


so the system actually extracts the speaker i.d. from the waveform names.


right i saw that.


and there's a there's a script and that is actually all in one script.


so there's this one script that parses waveform names.


and extracts things like the um speaker uh i.d.


or something that can stand in as a speaker i.d.


so we might have to modify that script to recognize the um speakers um in the in the uh um t.i. digits database.


right.


and that uh


or you can fake you can fake names for these waveforms that resemble the names that we use here for the for the meetings.


that would be the sort of probably the safest way to do


i might have to do that anyway to to do


because we may have to do an extract to get the amount of data per speaker about right.


uhhuh.


the other thing is isn't t.i. digits isolated digits?


or is that another one?


i'm i looked through a bunch of the digits corpora.


and now they're all blurring.


because one of them was literally people reading a single digit.


and then others were connected digits.


yeah most of t.i. digits is connected digits i think.


okay.


the i mean we had a bellcore corpus that we were using.


maybe it's the bell gram.


it was that's that was isolated digits.


bell digits.


all right.


by the way i think we can improve these numbers if we care to improve them by um not starting with the switchboard models but by taking the switchboard models and doing supervised adaptation on a small amount of digit data collected in this setting.


um


yep.


because that would adapt your models to the room acoustics.


and for the far field microphones you know to the noise.


and that should really improve things um further.


and then you use those adapted models which are not speaker adapted but sort of you know channel adapted.


channel adapted.


use that as the starting models for your speaker adaptation.


yeah but the thing is uh i mean when you it depends whether you're were just using this as a a starter task for you know to get things going for conversational or if we're really interested in connected digits.


and i i think the answer is both.


well i don't know.


and for for connected digits over the telephone you don't actually want to put a whole lot of effort into adaptation.


because somebody gets on the phone and says a number.


and then you just want it.


you don't don't uh


this is this that one's better.


right.


uhhuh.


um but you know i uh my impression was that you were actually interested in the far field microphone uh problem.


i mean


so


you want to you want to that's the obvious thing to try.


oh.


right?


then uh because you you don't have any


yeah.


that's where the most acoustic mismatch is between the currently used models and the the the set up here.


yeah so that'd be another interesting data point.


i mean i i guess i'm saying i don't know if we'd want to do that as the as


other way.


liz


now you're all watching me.


it it clips over your ears.


all right.


this way.


there you go.


if you have a strong if you have a strong preference you could use this.


you're all watching.


this is terrible.


it's just we we think it has some spikes.


so uh we we didn't use that one.


i'll get it.


but you could if you want.


yeah at any rate i don't know if


i don't know.


and andreas your your microphone's a little bit low.


it is?


yeah.


i don't know if we want to use that as the


uh


uh it pivots.


so if you see the picture


i


it it like this.


and then you have to


i i already adjusted this a number of times.


i i


yeah i think these mikes are not working as well as i would like.


can't quite seem to


yeah i think this contraption around your head is not working so well.


too many too many adjustments yeah.


anyway what i was saying is that i i think i probably wouldn't want to see that as sort of like the norm that we compared all things to.


that looks good.


to uh the to have have all this all this uh adaptation.


but i think it's an important data point if you're if


right.


yeah.


um


the other thing that that uh of course what barry was looking at was was just that.


the near versus far.


and yeah the adaptation would get some of that.


uhhuh.


but i think even even if there was uh only a factor of two or something like i was saying in the email i think that's that's a big factor.


so


liz you could also just use the other mike if you're having problems with that one.


well.


okay.


yeah this would be okay.


we we we think that this has spikes on it.


it's this thing's this is too big for my head.


so it's not as good acoustically.


yeah basically your ears are too big.


but


i mean mine are too.


everybody's ears are too big for these things.


no my my but this is too big for my head.


uh


so i mean it doesn't you know it's


well if you'd rather have this one then it's


yeah.


oh well.


it's great.


so the to get that uh pivoted this way it pivots like this.


no this way.


there you go.


and there's a screw that you can tighten.


and then it


right.


i already tried to get it close.


good.


so if it doesn't bounce around too much that's actually good placement.


that looks good.


okay.


but it looks like it's going to bounce a lot.


so where were we?


uh yeah.


yeah.


digits.


adaptation.


uh adaptation non adaptation.


um factor of two.


um


what by the way what factor of two did you


oh yeah i know what i was.


i mean


oh no no.


it's that that we were saying you know well is how much worse is far than near you know.


oh okay.


and i mean it depends on which one you're looking at.


that factor of two.


uhhuh.


but for the everybody it's little under a factor or two.


yeah i i know what i was thinking was that maybe uh we could actually try at least looking at uh some of the the large vocabulary speech from a far microphone.


at least from the good one.


i mean before i thought we'd get you know a hundred and fifty percent error or something.


but if if uh if we're getting thirty five forty percent or something um


actually if you run though on a close talking mike over the whole meeting during all those silences you get like four hundred percent word error.


right i understand.


but doing the same kind of limited thing


or or some high number.


yeah sure.


get all these insertions.


but i'm saying if you do the same kind of limited thing as people have done in switchboard evaluations or as


yeah.


where you know who the speaker is and there's no overlap?


and you do just the far field for those regions?


yeah the same sort of numbers that we got those graphs from.


could we do exactly the same thing that we're doing now but do it with a far field mike.


right?


yeah do it with one of


because we extract the times from the near field mike.


but you use the acoustics from the far field mike.


right i understand that.


i just meant that so you have three choices.


there's um you can use times where that person is talking only from the transcripts but the segmentations were were synchronized.


or you can do a forced alignment on the close talking to determine that you know within this segment these really were the times that this person was talking.


and elsewhere in the segment other people are overlapping.


and just front end those pieces.


or you can run it on the whole data.


which is which is you know a


but but but how did we get the how did we determine the links uh that we're testing on in the stuff we reported?


in the h.l.t. paper we took segments that are channel time aligned.


which is now being changed in the transcription process.


which is good.


and we took cases where the transcribers said there was only one person talking here.


because no one else had time any words in that segment.


and called that non overlap.


and


and that's what we were getting those numbers from.


yes.


right.


good the good numbers.


the bad numbers were from the segments where there was overlap.


well we could start with the good ones.


but anyway so i think that we should try it once with the same conditions that were used to create those.


yeah.


and in those same segments just use one of the p.z.m.'s.


right so we we can do that.


and then you know i mean the thing is if we were getting uh what thirty five forty percent something like that on on that particular set


uh does it go to seventy or eighty?


or does it use up so much memory we can't decode it?


it might also depend on which speaker it is and how close they are to the p.z.m.


uh


i don't know how different they are from each other.


you want to probably choose the p.z.m. channel that is closest to the speaker.


to be best


for this particular digit ones i just picked that one.


well


okay.


oh okay.


so we would then use that one too.


so


this is kind of central.


or


you know it's so but i would i'd pick that one.


it'll be less good for some people than for other.


but i i'd like to see it on the same exact same data set that that we did the other thing on.


actually


i actually should've picked a different one.


right?


because that could be why the p.d.a. is worse.


because it's further away from most of the people reading digits.


it's further away yeah.


yeah.


that's probably one of the reasons.


huh.


uhhuh.


well yeah you could look at i guess that p.z.m. or something.


yep.


but the other is it's very uh i mean even though there's i'm sure the the the s.r.i. uh front end has some kind of pre emphasis it's it's uh still it's picking up lots of low frequency energy.


so even discriminating against it i'm sure some of it's getting through.


um


but yeah you're right.


a part of it is just the distance.


and aren't these pretty bad microphones?


i mean


well they're bad.


but i mean if you listen to it it sounds okay you know.


yeah when you listen to it uh the p.z.m. and the p.d.a. yeah the p.d.a. has higher sound floor.


but not by a lot.


it's really pretty uh pretty much the same.


i just remember you saying you got them to be cheap on purpose.


cheap in terms of their quality.


so


well they're twenty five cents or so.


we wanted them to be to be typical of what would be in a p.d.a.


yeah.


so they are they're not the p.z.m. three hundred dollar type.


uhhuh.


they're the twenty five cent.


buy them in packs of thousand type.


but i mean the thing is people use those little mikes for everything.


i see.


everything.


because they're really not bad.


i mean if you're not doing something ridiculous like feeding it to a speech recognizer they they they you know you can hear the hear the sounds just fine.


right.


you know it's


they i mean it's more or less the same principles as these other mikes are built under.


it's just that there's less quality control.


they just you know churn them out and don't check them.


um so


so that was yeah so that was interesting result.


so like i said the front end guys are very much interested in in this is as as well.


and


so so but where is this now?


i mean what's where do we go from here?


i mean


yeah that was going to be my question.


we so we have a we have a a system that works pretty well.


but it's not you know the system that people here are used to using to working with.


so what what do we do now?


well i think what we want to do is we want to uh


and we've talked about this in other contexts.


we want to have the ability to feed it different features.


uhhuh.


and then um from the point of view of the front end research it would be uh substituting for h.t.k.


okay.


i think that's the key thing.


and then if we can feed it different features then we can try all the different things that we're trying there.


okay all right.


and then um uh also dave is is thinking about using the data in different ways uh to um uh explicitly work on reverberation.


starting with some techniques that some other people have found somewhat useful and yeah.


so so the key thing that's missing here is basically the ability to feed you know other features into the recognizer.


and also then to train the system.


right.


and uh i don't know when chuck will be back.


but that's exactly what he he's going to


he's he's sort of back.


but he drove for fourteen hours and wasn't going to make it in today.


oh okay.


so i think that's one of the things that he said he would be working on.


um


just sort of to make sure that we can do that.


yeah.


and um


it's uh i mean the the front end is that's in the s.r.i. recognizer is very nice in that it does a lot of things on the fly.


but it unfortunately is not designed and um like the uh icsi system is where you can feed it from a pipeline of of the command.


so the what that means probably for the foreseeable future is that you have to uh dump out um


you know if you want to use some new features you have to dump them into individual files.


and give those files to the recognizer.


we do we tend to do that anyway.


okay.


oh.


so although you you can pipe it as well we tend to do it that way.


because that way you can concentrate on one block and not keep redoing it over and over.


all right.


yeah so i've i


so that's exactly what the p. file is for.


yeah the the the cumbersome thing is is um is that you actually have to dump out little little files.


uh


so for each segment that you want to recognize you have to dump out a separate file.


uhhuh.


just like like as if there were these waveform segments.


but instead you have sort of feature file segments.


but you know


so


cool.


okay so the the the next thing we had on the agenda was something about alignments?


yes we have


i don't know.


did you want to talk about it?


or


i can give a i was just telling this to jane.


and and we we were able to get some definite improvement on the forced alignments by looking at them first and then realizing the kinds of errors that were occurring.


and um


some of the errors occurring very frequently are just things like the first word being moved to as early as possible in the recognition.


which is a um i think was both a a pruning problem.


and possibly a problem with needing constraints on word locations.


and so we tried both of these things.


we tried saying


i got this whacky idea that just from looking at the data that when people talk their words are usually chunked together.


it's not that they say one word and then there's a bunch of words together.


they're might say one word and then another word far away if they were doing just backchannels.


but in general if there's like five or six words and one word's far away from it that's probably wrong on average.


so um


and then also the pruning of course was too too severe.


so that's actually interesting.


the pruning was the same value that we used for recognition.


and we had lowered that we had used tighter pruning after liz ran some experiments showing that you know it runs slower.


and there's no real difference in


no gain.


actually it was better with slightly better or about


right.


it was the same with tighter pruning.


so for free recognition this the lower pruning value is better.


you


it's probably because the recognition's just bad at a point where it's bad enough that that you don't lose anything.


correct.


um but it turned out for for to get accurate alignments it was really important to open up the pruning significantly.


huh.


um because otherwise it would sort of do greedy alignment um in regions where there was no real speech yet from the foreground speaker.


uhhuh.


um so that was one big factor that helped improve things.


and then the other thing was that


you know as liz said the we enforce the fact that uh the foreground speech has to be continuous.


it cannot be you cannot have a background speech hypothesis in the middle of the foreground speech.


you can only have background speech at the beginning and the end.


yeah i mean yeah it isn't always true.


and i think what we really want is some clever way to do this.


where um you know from the data or from maybe some hand corrected alignments from transcribers that


things like words that do occur just by themselves alone like backchannels or something that we did allow to have background speech around it


yeah.


those would be able to do that.


sorry.


but the rest would be constrained.


so i think we have a version that's pretty good for the native speakers.


i don't know yet about the non native speakers.


and um


we basically also made noise models for the different sort of grouped some of the mouth noises together.


um so and then there's a background speech model.


and we also


there was some neat or interesting cases.


like there's one meeting where um jose's giving a presentation.


and he's talking about um the word mixed signal.


and someone didn't understand uh that you were saying mixed.


i think morgan.


yeah yeah.


and so your speech was saying something about mixed signal.


and the next turn was a lot of people saying mixed.


like he means mixed signal or i think it's mixed.


and the word mixed in this segment occurs like a bunch of times.


and chuck's on the lapel here.


and he also says mixed.


but it's at the last one.


and of course the aligner aligns it everywhere else to everybody else's mixed.


yeah.


because there's no adaptation yet.


so there's i think there's some issues about


we probably want to adapt at least the foreground speaker.


but i guess andreas tried adapting both the foreground and a background generic speaker.


and that's actually a little bit of a funky model.


like it gives you some weird alignments.


just because often the background speakers match better to the foreground than the foreground speaker.


oh


oh.


so there's some things there.


especially when you get lots of the same words uh occurring in the.


well the


i i think you can do better by uh cloning


so we have a reject phone.


and you and what we wanted to try with you know once we have this paper written and have a little more time uh cloning that reject model.


and then one copy of it would be adapted to the foreground speaker to capture the rejects in the foreground.


like fragments and stuff.


and the other copy would be adapted to the background speaker.


right i mean in general we actually


and


right now the words like partial words are reject models.


uhhuh.


and you normally allow those to match to any word.


but then the background speech was also a reject model.


and so this constraint of not allowing rejects in between.


you know it needs to differentiate between the two.


so just sort of working through a bunch of debugging kinds of issues.


right.


and another one is turns like people starting with well i think


and someone else is well how about


so the word well is in this in this segment multiple times.


and as soon as it occurs usually the aligner will try to align it to the first person who says it.


but then that constraint of sort of uh proximity constraint will push it over to the person who really said it in general.


is the proximity constraint a hard constraint?


or did you do some sort of probabilistic weighting distance or


we we didn't


no.


right now it's a kluge.


we


okay.


we it's straightforward to actually just have a a penalty that doesn't completely disallows it but discourages it.