Pitch adjustment #329

ryanheise · 2021-03-03T10:14:24Z

Is your feature request related to a problem? Please describe.

Pitch adjustment would be useful in some apps (e.g. changing pitch of voice or music).

Describe the solution you'd like

player.setPitch(1.0); // 1.0 being normal pitch

Describe alternatives you've considered

There is no other way to achieve this other than plugin support.

Additional context

This is easily implemented on Android, iOS might be more problematic (post below if you are able to share useful resources/documentation)

nt4f04uNd · 2021-03-03T10:33:32Z

i think you should add it to android and then some time later implement it on iOS

ryanheise · 2021-03-03T11:10:13Z

True.

I should probably mention that all new features from this point will only be developed on the null-safe branch (i.e. the 0.7.* releases). For now, this will require the beta channel of the Flutter SDK, although stable is coming soon.

ryanheise · 2021-03-03T11:41:57Z

Note to self. This API might integrate well with AVQueuePlayer:

https://developer.apple.com/documentation/avfoundation/avaudiomixinputparameters/1387042-audiotimepitchalgorithm?language=objc

(Though I'm not particularly happy with the quality of iOS's pitch/time stretching algorithms, if the above works, I may as well use it.)

leidig54 · 2021-03-04T10:57:59Z

This would be another massive winner for me. Any eta? As with the equalizer functionality, happy to offer a bounty to bump the priority.

ryanheise · 2021-03-06T14:57:37Z

Hi @leidig54

Thanks again for offering to support the project!

I will ordinarily prioritise features based on the number of thumbs up and their difficulty, although I also tend to prioritise requests from GitHub sponsors too (otherwise, you're welcome to discuss by email - the address can be found in the git log).

The next two big features I plan to work on are the visualizer and equalizer, but I may throw the pitch shifting feature into that group, too, since there may be some overlap in the section of code involved on the iOS side.

ryanheise · 2021-03-10T02:47:43Z

It definitely has overlap with what's happening on the visualizer branch and I want to ensure I get the overall design right in how these features interact, so I'm looking into this now.

ryanheise · 2021-03-10T03:40:05Z

As I originally suspected, this feature would best be handled by a move to AVAudioEngine on iOS.

Although there might be ways to get this to work with AVQueuePlayer, maybe it is finally time to make the switch to AVAudioEngine, given that a lot of the popular feature requests at the moment involve audio processing. This would also allow complete control over audio interruptions since AVQueuePlayer is hardcoded to take over this responsibility.

There are some existing libraries built over the top of AVAudioEngine that could make things easier, which I will have a look at first. For example, AudioKit seems to provide everything we need in terms of audio processing, and there are others (and there's the option of just using the AVAudioEngine API directly), but I will just need to evaluate these options based on how well they support the complete just_audio feature set.

I'd estimate the rewrite of existing features at maybe 1-to-2 months at a minimum, if worked on full time, after which I would expect pitch shifting to be an extra couple of days on top of that.

Given that, I think I may go ahead and release the feature for Android first.

nt4f04uNd · 2021-03-10T03:47:34Z

i really love the direction you are taking in working on your audio plugins, and how you implement features. these are by far the most mature and will-maintained flutter plugins

ryanheise · 2021-03-10T09:12:16Z

The Android implementation is now available for testing on the pitch branch.

ryanheise · 2021-03-12T03:04:41Z

For the first time with this project I'm starting to feel the need for a dev branch (we have a number of features that will need to be tested together before being stable) - I'll do some restructuring.

afkcodes · 2021-03-12T07:29:09Z

@ryanheise completely agreed with you here, just a suggestion we can also update the readme for the new features in branch. Also eady to support the project, for the good work you are doing.

ryanheise · 2021-03-13T05:15:28Z

dev is now created. It contains the new pitch feature (Android only so far) and also #300 (buffer configuration options). My immediate goal is to bunch together the features that will involve changes to the platform interface so that I can do a single release (because the release process for changes to the platform interface is usually time consuming.) I will probably also include in this bunch the "skip silence" feature (just Android for now). Once that's all done, I can focus on the iOS implementation.

ryanheise · 2021-04-01T00:39:19Z

(Copying this comment from another issue to get broader interest)

Is this supported on iOS currently?

The waveform visualizer is implemented on iOS but not pitch. You can track the pitch feature here: #329

There is a big question at this point whether to continue with the current AVQueuePlayer-based implementation or switch to an AVAudioEngine-based implementation. For pitch scaling, I really want to take advantage of AVAudioEngine's built-in features, but that requires a rewrite of the iOS side - see #334 and this is a MUCH bigger project.

I would really like to see an AVAudioEngine-based solution see the light of day, but it will probably not happen if I work on it alone. If anyone would like to help, maybe we can pull it off with some solid open source teamwork. One of the attractive solutions is to use AudioKit which is a library built on top of AVAudioEngine which also provides access to pitch adjustment AND provides a ready-made API for a visualizer and equalizer. That is, it provides us with everything we need - BUT it is written in Swift and so that involves a language change and it means we may need to deal with complaints that old projects don't compile (we'd need to provide extra instructions on how to update their projects to be Swift-compatible).

Would anyone like to help me with this? (Please reply on #334)

smakarov · 2021-04-06T09:55:27Z

Does it possible to use BASS or Superpowered(best for pitch shifting) or you would like to implement this by yourself?

ryanheise · 2021-04-06T10:07:26Z

@smakarov this should be relatively straightforward to do once the AVAudioEngine / AudioKit implementation is done. See:

https://audiokit.io/playgrounds/Playback/Time%20Stretching%20and%20Pitch%20Shifting/

So the main effort will go into #334 itself.

akashrajkn · 2021-05-21T18:07:28Z

Is it also possible to get the pitch of recorded audio?

BTW, this plugin very helpful, thanks for developing it :)

ghost · 2021-06-20T14:15:07Z

Hi,
is there any update for this feature?

ryanheise · 2021-06-20T14:17:48Z

@gkmuhannad this is already implemented on Android (on the dev branch), iOS will take quite a while longer. I will publish a release including the Android implementation as soon as I update the unit tests.

ghost · 2021-06-20T14:19:21Z

Great😍.. Thank you so much Get Outlook for iOS<https://aka.ms/o0ukef>

…

________________________________ From: ryanheise ***@***.***> Sent: Sunday, June 20, 2021 5:18:01 PM To: ryanheise/just_audio ***@***.***> Cc: Almuhannad Ghaleb ***@***.***>; Mention ***@***.***> Subject: Re: [ryanheise/just_audio] Pitch adjustment (#329) @gkmuhannad<https://github.com/gkmuhannad> this is already implemented on Android (on the dev branch), iOS will take quite a while longer. I will publish a release including the Android implementation as soon as I update the unit tests. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#329 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ATI3EV23IRT7QN47EVWTCTLTTX2BTANCNFSM4YQ52OGQ>.

ryanheise · 2021-06-21T15:46:15Z

The Android implementation is now published in release 0.8.0.

geekmuhannad · 2021-07-03T18:49:20Z

@ryanheise
Hello bro
is there any way to save the audio after applying the pitch ?

demberto · 2021-08-31T05:33:44Z

Always happy to help

kekko7072 · 2022-03-24T21:23:49Z

Any update on this topic? I need a package to manage pitch and tune, any suggestion?

ryanheise · 2022-03-25T06:19:01Z

There is some progress being made in #334 on an AVAudioEngine-based implementation. This should make it much easier to implement this functionality on iOS.

Some notes for implementation, these are the classes to look at to adjust the speed and pitch of audio:

AVAudioUnitVarispeed
AVAudioUnitTimePitch

watrindade · 2022-04-06T22:24:16Z

@ryanheise I'm having trouble trying to use it in the version 0.9.20. Is there a way to raise a song pitch just one half step? I mean, a song in C goes to C#?

Thank you the amazing work you have been doing to the community. We very much appreaciate it.

ryanheise · 2022-04-07T02:35:37Z

I guess you want to know the formula to map a semitone into a pitch adjustment scale. I don't actually know, because the underlying native API just says this scale is "the factor by which pitch will be shifted". An answer to this question will depend on the answer to the same question for native app developers, so I would suggest Googling for that. And if you don't find anything, I would suggest doing a little bit of experimentation and mathematics. E.g. what does a factor of 2x give you? Is it just doubling the frequency? If so, that's a starting point for your calculation.

akshdeep-singh · 2022-08-09T16:59:56Z

@ryanheise the pitch adjustment doesn't work on android when using just_audio_background? It shows UnimplementedError, but it worked for me before using background plugin.

ryanheise · 2022-08-10T02:51:44Z

Thanks for reporting, @akshdeep-singh . Actually there are several methods not implemented by just_audio_background:

androidEqualizerBandSetGain
androidEqualizerGetParameters
androidLoudnessEnhancerSetTargetGain
audioEffectSetEnabled
setCanUseNetworkResourcesForLiveStreamingWhilePaused
setPitch
setPreferredPeakBitRate
setSkipSilence

I'll implement these at my next opportunity.

Jeff09github · 2022-09-11T16:09:27Z

any update for web platform?

ryanheise · 2022-09-11T16:14:14Z

It's one of the next items on my list, although my main computer has been down so not yet.

ryanheise · 2022-09-15T16:05:54Z

Sorry I meant to say that fixing setPitch in just_audio_background was my next item. For web, it is not a priority, although contributions are welcome (it will require a different web implementation based on web audio).

@akshdeep-singh I have fixed the unimplemented methods in just_audio_background now. Can you try out the fix in the fix/unimplemented_methods branch and let me know if it is working for you?

ryanheise · 2022-09-16T03:46:38Z

Regarding fix/unimplemented_methods, I have only tested setPitch although the other methods in the above list should also work now (audio effects, skip silence, etc.)

akshdeep-singh · 2022-09-20T21:11:17Z

@ryanheise I have verified pitch change, it's working for me. Thanks for your support.

aamironline · 2023-03-17T18:37:53Z

Thank you very much for creating such a great library. I would like to know the current status of this issue. Could anyone please update? Thanks again.

ryanheise · 2023-03-18T05:33:18Z

I promise that my comments above truthfully reflect the status, as does the README page.

alexgrusu · 2023-12-12T12:33:56Z

I can see setPitch is called in setPlatform.

I'm running an app on an iOS simulator and I'm encountering this issue

[ERROR:flutter/runtime/dart_vm_initializer.cc(41)] Unhandled Exception: PlatformException(-1008, resource unavailable, null, null)

It seems the issue occurs due to the setPitch call.

Is there a way to avoid the setPitch call?

ryanheise · 2023-12-12T13:00:00Z

Since the platform support is not equal, you can write code like this:

if (Platform.isAndroid) {
    await _player.setPitch(pitch);
}

If you're also cross compiling to web, then you can do this:

if (!kIsWeb && Platform.isAndroid) {
    await _player.setPitch(pitch);
}

alexgrusu · 2023-12-12T13:05:42Z

Since the platform support is not equal, you can write code like this:
if (Platform.isAndroid) {
    await _player.setPitch(pitch);
}
If you're also cross compiling to web, then you can do this:
if (!kIsWeb && Platform.isAndroid) {
    await _player.setPitch(pitch);
}

I'm not calling setPitch in my code. setPitch is called by setPaltform.

ryanheise · 2023-12-12T13:12:27Z

There is the following code in setPlatform:

        try {
          await platform.setPitch(SetPitchRequest(pitch: pitch));
        } catch (e) {
          // setPitch not supported on this platform.
        }

So here the exception is handled. Are you sure that this is the call site that is causing the issue? Maybe you're actually using a different plugin such as just_audio_background? Can you reproduce this with one of the official example project, and if so, can you tell me which one and how to reproduce it?

alexgrusu · 2023-12-12T13:54:58Z

There is the following code in setPlatform:
        try {
          await platform.setPitch(SetPitchRequest(pitch: pitch));
        } catch (e) {
          // setPitch not supported on this platform.
        }
So here the exception is handled. Are you sure that this is the call site that is causing the issue? Maybe you're actually using a different plugin such as just_audio_background? Can you reproduce this with one of the official example project, and if so, can you tell me which one and how to reproduce it?

I'm using just_audio only at the moment.

I tried the official example and I modified the line where the audio source is set to

await _player.setUrl();

and I'm encountering the same error

flutter: Error loading audio source: (-1008) resource unavailable

There might be an issue on my side because the actual URL doesn't contain the file extension and I added the extension to the URL (I mean in the Flutter app)

Is there a way to specify the file extension?

LE:
because the URL is something like this: https://erxample/path/file

I added m4a to the end https://erxample/path/file.m4a. I know on iOS the extension is required

ryanheise · 2023-12-12T13:58:44Z

@alexgrusu Would you mind opening a new bug report? I don't think this issue is specifically related to the setPitch implementation, and if there is a bug somewhere in the setPlatform code, then I would like to investigate that separately from this issue about the setPitch feature.

alexgrusu · 2023-12-12T14:46:23Z

@alexgrusu Would you mind opening a new bug report? I don't think this issue is specifically related to the setPitch implementation, and if there is a bug somewhere in the setPlatform code, then I would like to investigate that separately from this issue about the setPitch feature.

Sounds good, I will open a different bug.

Might be my approach wrong? I mean, setting the m4a extension manually (the extension is not provided by the URL), could lead to an exception? Is there a way of setting the extension of the file when the link doesn't contain the extension?

araa12 · 2024-02-20T22:01:16Z

any update of pitch change on iOS?

ryanheise · 2024-02-21T01:21:48Z

This is something that would require an AVAudioEngine-based implementation. You could take a look at #784 as a starting point, and consider contributing to it to add pitch adjustment.

GayakEngineer · 2024-07-08T12:55:07Z

@ryanheise Hello. I could not find any documentation on setPitch(). What do i have to pass as argument to setPitch() ? What would be the factor, in order to shift pitch by one semitone.

In android flutter app, just audioplayer_name.setPitch(pitch); is enough, am i right?

ryanheise · 2024-07-08T13:03:25Z

@GayakEngineer

Hello. I could not find any documentation on setPitch().

It should be there on https://pub.dev/packages/just_audio under the link "API Documentation". Find the class (i.e. AudioPlayer), then find the thing inside the class (i.e. setPitch). The README page gives the supported platforms (as mentioned above, only Android).

In android flutter app, just audioplayer_name.setPitch(pitch); is enough, am i right?

That same comment above confirms this is correct.

GayakEngineer · 2024-07-08T13:14:38Z

@GayakEngineer

Hello. I could not find any documentation on setPitch().

It should be there on https://pub.dev/packages/just_audio under the link "API Documentation". Find the class (i.e. AudioPlayer), then find the thing inside the class (i.e. setPitch). The README page gives the supported platforms (as mentioned above, only Android).

In android flutter app, just audioplayer_name.setPitch(pitch); is enough, am i right?

That same comment above confirms this is correct.

Thank you for such a prompt reply. You have done great service to flutter community.

GayakEngineer · 2024-08-05T07:28:16Z

@ryanheise any update on setPitch for iOS?

ryanheise · 2024-08-05T07:37:31Z

The iOS side depends on rewriting using AVAudioEngine (see #334 for further details)

ryanheise added enhancement New feature or request 1 backlog labels Mar 3, 2021

ryanheise self-assigned this Mar 3, 2021

ryanheise added 2 fixing and removed 1 backlog labels Mar 10, 2021

ryanheise mentioned this issue Mar 10, 2021

Rewrite iOS implementation based on AVAudioEngine #334

Open

ryanheise mentioned this issue Mar 15, 2021

Casting support #211

Open

ryanheise mentioned this issue Apr 1, 2021

Provide a simple Audio Visualizer #97

Open

ryanheise mentioned this issue Apr 1, 2021

Equalizer #147

Open

This comment has been minimized.

Sign in to view

Pitch adjustment #329

Pitch adjustment #329

Comments

ryanheise commented Mar 3, 2021

nt4f04uNd commented Mar 3, 2021 • edited Loading

ryanheise commented Mar 3, 2021

ryanheise commented Mar 3, 2021

leidig54 commented Mar 4, 2021

ryanheise commented Mar 6, 2021

ryanheise commented Mar 10, 2021

ryanheise commented Mar 10, 2021

nt4f04uNd commented Mar 10, 2021

ryanheise commented Mar 10, 2021

ryanheise commented Mar 12, 2021

afkcodes commented Mar 12, 2021 • edited Loading

ryanheise commented Mar 13, 2021

ryanheise commented Apr 1, 2021

smakarov commented Apr 6, 2021

ryanheise commented Apr 6, 2021

akashrajkn commented May 21, 2021 • edited Loading

ghost commented Jun 20, 2021

ryanheise commented Jun 20, 2021

ghost commented Jun 20, 2021 via email

ryanheise commented Jun 21, 2021

geekmuhannad commented Jul 3, 2021

demberto commented Aug 31, 2021

This comment has been minimized.

This comment has been minimized.

kekko7072 commented Mar 24, 2022

ryanheise commented Mar 25, 2022

watrindade commented Apr 6, 2022

ryanheise commented Apr 7, 2022

akshdeep-singh commented Aug 9, 2022

ryanheise commented Aug 10, 2022

Jeff09github commented Sep 11, 2022

ryanheise commented Sep 11, 2022

ryanheise commented Sep 15, 2022

ryanheise commented Sep 16, 2022

akshdeep-singh commented Sep 20, 2022

aamironline commented Mar 17, 2023

ryanheise commented Mar 18, 2023

alexgrusu commented Dec 12, 2023 • edited Loading

ryanheise commented Dec 12, 2023

alexgrusu commented Dec 12, 2023

ryanheise commented Dec 12, 2023

alexgrusu commented Dec 12, 2023 • edited Loading

ryanheise commented Dec 12, 2023

alexgrusu commented Dec 12, 2023

araa12 commented Feb 20, 2024

ryanheise commented Feb 21, 2024

GayakEngineer commented Jul 8, 2024 • edited Loading

ryanheise commented Jul 8, 2024

GayakEngineer commented Jul 8, 2024

GayakEngineer commented Aug 5, 2024

ryanheise commented Aug 5, 2024

nt4f04uNd commented Mar 3, 2021 •

edited

Loading

afkcodes commented Mar 12, 2021 •

edited

Loading

akashrajkn commented May 21, 2021 •

edited

Loading

alexgrusu commented Dec 12, 2023 •

edited

Loading

alexgrusu commented Dec 12, 2023 •

edited

Loading

GayakEngineer commented Jul 8, 2024 •

edited

Loading