Secrets of the MediaPlayer framework for iOS

1394 Views

For those who have ever tried developing a music player app, you've probably been left baffled my the MediaPlayer framework at least a few times. You've probably also seen popular apps such as Cs Music Player (formerly Cesium) and wondered how they can get certain information (how do you know it's a playlist folder?) that is not publicly documented. If you're wondering these things, you're in the right place. I should note--the methods used below are undocumented and may stop working at one point or another. Alright, let's get started!

Playlist Folders

To know if an MPMediaPlaylist object is a playlist folder or not, simply do the following:

let playlistIsAFolder = playlist.value(forProperty: "isFolder") as? Bool 

Next, let's retrieve all the playlists that belong in this playlist folder:

extension MPMediaPlaylist {
    var folderItems: [MPMediaPlaylist] {
        var allFolderItems = [MPMediaPlaylist]()
        if let playlists = MPMediaQuery.playlists().collections as? [MPMediaPlaylist], let id = value(forProperty: "persistentID") as? Int  {
            for playlist in playlists {
                if let parentId = playlist.value(forProperty: "parentPersistentID") as? Int {
                    if parentId != 0 && parentId == id {
                        allFolderItems.append(playlist)
                    }
                }
            }
        }
        return allFolderItems
    }
}
Here, we are assuming that the MPMediaPlaylist object we are dealing with is a playlist folder, and we're getting all the playlists that belong in this folder. So, we loop through all other playlists and see if their "parentPersistentID" property corresponds to the persistentID property of the playlist folder. If a playlist does not belong to any playlist folder, its parentId is 0, which is why we filter out playlists whose parentPersistentID is 0.

Getting loved/disliked state (for Apple Music items)

In Apple Music, you can love/dislike a song. If you want to know whether a song is loved or disliked, simply do the following for an MPMediaItem object:
if let likedState = mediaItem.value(forProperty: "likedState") as? Int {
    // likedState == 2 means song is Loved
    // likedState == 3 means disliked
    // likedState == 1 means neither liked nor disliked
}

Global Cloud ID and Share URL for playlists

Not all playlists will have this property. Only playlists that are either 1) Apple Music curated playlists 2) user playlists that were created on an iPhone (not on your computer with iTunes) or 3) have been shared will have this property. So, if you create a playlist on your computer, and then run this code on your iPhone, you will not get its share URL or cloud ID (since they don't exist yet). Here's the code to get each (assuming the playlist is an MPMediaPlaylist object):

if let globalID = playlist.value(forProperty: "cloudGlobalID") as? String {
    if globalID.count > 0 {
        // do something with global ID
    }
}

Also, to note: the globalID property will never actually be nil. If it doesnt exist, it will just be an empty string. That's why I test for globalID.count > 0

To get the share URL of a playlist:

if let shareURL = playlist.value(forProperty: "cloudShareURL") as? String {
   if shareURL.count > 0 {
        // do something with shareURL
    }
}

Getting playlist type

I will forewarn that this method is a bit hacky, but it seems to work quite reliably as far as I can tell. So the idea is, for each playlist, it can fall into several different types: Is it an Apple Music playlist (ie, Lana del Rey essentials, Next Steps, etc), is it a playlist a user shared with you, or is it one of your own playlists? We will use a combination of undocumented properties to extract this info.

Detecting Apple Music Playlist

To detect if it's an Apple Music playlist, we can look at is globalID as we got in the example above. Apple Music playlists have a global ID of format pl.xxxxxxx, and user generated playlists have an ID of format pl.u-xxxxx.

User generated -- is it my playlist, or one shared with me from my friend?

For this, we need to use the property isEditable:

 let playlistIsEditable = playlist.value(forProperty: "isEditable") as? Bool

This works because at least as of iOS 13, Apple Music does not have collaborative playlists, and users cannot edit other users' playlists. Therefore, if you have a playlist that you know is not an Apple Music playlist, and also cannot be edited by you--then you can deduce that it is a playlist shared with you by your friend.

Loving/Disliking a song on Apple Music

We need to use the Apple Music API to love or dislike a local library song. however, the ID that we need to do so is not the MPMediaItem persistentID that comes as the ID of all local songs. We need an ID in the format of i.xxxxxxxx (as seen here: https://developer.apple.com/documentation/applemusicapi/add_a_personal_library_song_rating). To get this ID, just do this:

if let libraryId = mediaItem.value(forProperty: "cloudUniversalLibraryID") as? String {
    // do something with the library Id
}

Get Date Added of MPMediaItem

You're probably thinking--but it already has a dateAdded property! And it does, and in true MediaPlayer framework fashion, it doesnt work 100% of the time. The problem is that actually, the dateAdded property is incorrectly typed, and it should be optional, as not all MPMediaItems have this property. 99.9% do, but when 1 doesnt, you will get crash reports from beta testers as I did, that your app is crashing on launch. So, instead of calling .dateAdded, do this:

extension  MPMediaItem {
    func getAddedDate() -> Date? {
        return self.perform(#selector(getter: MPMediaItem.dateAdded))?.takeUnretainedValue() as? Date
    }
}

if let dateAdded = mediaItem.getAddedDate() {
    // now it's safe to use
}

Get Album Artwork for Now Playing song

In most cases this is easy, you just use MPMediaItem's self.artwork?.image(at: CGSize) method to get the artwork. However, this is not possbile in all cases, as not all album artwork has been downloaded. Also, What about songs streaming from Apple Music or songs from Apple Music Radio? I will note that the method I'm about to show only seems to work after the MPMediaItem song has been playing for about 2 seconds or so--which means if you change the song and check these properties immediately after, they will probably return nil. I'm not sure why it takes some time, but for whatever reason, it does. Anyways, here is how to get the URL for the artwork:

extension MPMediaItem {
    var imageURL: String? {
        if let artworkCatalog = self.value(forKey: "artworkCatalog") as? NSObject,
            let token = artworkCatalog.value(forKey: "token") as? NSObject {
            return token.value(forKey: "fetchableArtworkToken") as? String
        }
        return nil
    }
}

This is making use of a lot of undocumented properties, and there is no guarantee that they will work in the future, so I wouldnt recommend using this as a big part of your music app. I think it's a good last resort option if you have failed to get the artwork URL in other ways, but definitely should not be your first option. If you are using Music Kit and allowing users to browse Apple Music playlists, a better way would be to cache all the image URLs in UserDefaults that come from there so that you can easily look them up by key-value association when you need them later, instead of relying on something like this all the time. However, for some things (such as listening to Apple Music Radio), this undocumented way is the only option.

Retrieving non-library items

You may want to perform an MPMediaQuery to find non-library items, such as recently played songs (including those streamed from Apple Music), or playlists that you used to have but deleted. To do so, we will call the undocumented selector setShouldIncludeNonLibraryEntities, like so:

let query = MPMediaQuery.songs()
query.perform(Selector("setShouldIncludeNonLibraryEntities:"), with: true)
	

It’s important to note that this doesn’t work 100% of the time—for whatever reason, it works probably 75-80% of the time. Of course we are dealing with undocumented private APIs here, so it’s not surprising that the methods can be a bit unreliable.

Another way to retrieve recently played songs, including non-library songs is by searching playback history playlists. The following method retrieves all items from recently played playlists:

let query = MPMediaQuery.playlists()

query.perform(Selector("setShouldIncludeNonLibraryEntities:"), with: true)
   
var allHistoryPlaylists = [MPMediaPlaylist]()

let playlists = query.collections as? [MPMediaPlaylist] ?? []
for i in playlists {
    let isHistoryPlaylist = i.value(forProperty: "isPlaybackHistoryPlaylist") as? Bool ?? false
    if isHistoryPlaylist && i.items.count > 0 {
        allHistoryPlaylists.append(i)
    }
}
        
let allPlaybackHistoryItems = allHistoryPlaylists.flatMap { $0.items }.filter { $0.lastPlayedDate != nil }

So here we perform a query on playlists, and we use the undocumented property isPlaybackHistoryPlaylist to determine if the playlist is a playback history playlist. The system, for whatever reason, has items in these playlists with both nil lastPlayedDate properties, and playlists with 0 items, so we need to do a bit of filtering before we can get a flat, clean array. Even the allPlaybackHistoryItems object will have duplicate items and some items with lastPlayedDates that are quite old (like 1+yr), so you will need to further filter those on your own. So basically the system gives you what you need via this method, but also gives you a lot of junk that you need to first filter out.

Getting custom playlist artwork

This is really sketchy as it requires hardcoding part of a URL, however it does seem to work consistently in my experience. Just use the following extension:

extension MPMediaPlaylist {
    var artworkURL: String? {
        if let catalog = value(forKey: "artworkCatalog") as? NSObject, let token = catalog.value(forKey: "token") as? NSObject, let url = token.value(forKey: "availableArtworkToken") as? String {
            return "https://is2-ssl.mzstatic.com/image/thumb/\(url)/260x260cc.jpg"
        }
        return nil
    }
}

So yes, this is a bit sketchy, but it works. I think a better approach though is to get the user’s playlists from the Apple Music API, as it has the playlist artwork URL in that data as well. The only downside to that is the playlists with no custom artwork also have a URL property, which is the url of the blank iTunes icon, and nobody wants to see that.

Personally, what I do in my app is I use the above extension to determine if the playlist has custom artwork (by knowing if the url property is nil or not), and if it does, then I get the correct, cached URL that I got from Apple Music. If it does not have custom artwork, I just get an MPMediaItem from the playlist and use that as the artwork to display instead of the blank iTunes icon. This way, I don’t rely on the hard coded URL to work 100%, I only rely on the values themselves existing, and is thus bit safer. It is worth noting though, that the Apple Music API does not work with smart playlists, so any smart/genius mixes the user has cannot have their playlist artwork accessed via the Apple Music API—you must rely on the undocumented way above and risk it with the hard coded URL.

Anyways that’s it for this post. If you are writing a music related app, I hope you can find this information useful!